The Joel on Software Discussion Group (CLOSED)

A place to discuss Joel on Software. Now closed.

This community works best when people use their real names. Please register for a free account.

Other Groups:
Joel on Software
Business of Software
Design of Software (CLOSED)
.NET Questions (CLOSED)
Fog Creek Copilot

The Old Forum

Your hosts:
Albert D. Kallal
Li-Fan Chen
Stephen Jones

Limitations of Web Services

Say you had an idea for a web service that would require a ton of data to be shipped back and forth -- on the order of 1-5MB up to the service, with around a MB coming back down.  Is this a rational kind of use case for a typical SOAP/WSDL web service?

I only ask because the only successful (i.e. widely-used) web services I've seen so far have been RPC-ish things like google/yahoo search results, amazon listings, weather reports, etc.

Thursday, February 16, 2006
Well, it may not be feasible with RPC-style SOAP services, if only from a user point of view (why's this taking so long?). 

However, I work in insurance, and the major companies in the UK routinely use document-based web services to transfer documents/videos etc between themselves when documenting claims etc.

Most of the logic is based around remote searches of document repositories etc, but the same web service is used to actually transfer the data itself (it's limited to SwA / MIME at the moment, but will be upgraded to MTOM next year). 

There's a generally agreed limit of 10Meg, but this is more to accommodate mainframe back offices than for any SOAP-related reason.


Friday, February 17, 2006
Good question. About 9 months ago I was wondering the same myself.  The answer to your question is: Yes.

I've recently built out a web service application that transfers large amounts of data.  In my case it is a large, one way transfer of data with the client waiting for a response (message and return code) to come back, e.g. synchronous communication.  Large, to date has been up to 17mb of xml data. 

It is a Microsoft .Net application that has replaced a mainframe/COBOL application that would transfer fixed width, flat files.

This service is for batch data processing (as in server to server) and the web service is consumed among divisions within a large company.  Off the top of my head a 4mb transfer took well under a minute and most of that time was processing the data (validating and writing it to a database)

Additionally, the implementation of xml schemas has been a huge, A+ benefit. 

Even with the additional overhead of SSL and client certificate validation, performance has not been an issue.  Just be sure to increase your timeout and request payload limits appropriately.  Load balancing is key too, of course, depending on your requirements.
Green Eggs and Ham Send private email
Friday, February 17, 2006
It may make sense if you control the network.  In that case you are not at the mercy of the Internet and 4 megs goes by in about 2 seconds.

Even so, that sounds like a lot of data.  Now if it is one entity (say an audio or video file), that is not such a big deal.  However, if it is application data, you may want to question why you are shipping it around. 

An example:  client number, name, address,city,state,zip, country, ...add a bunch of fields.

I ask for Client 12345 as part of client maintenance.
You give me all the data for client 12345. You can either have me send back _everything_ or only items that changed. Some people in an effort to be purists will say "always send everything back."  Those of us who support distant clients recognize this can be an issue. (Four megs through ISDL to Tokyo can seem like forever).  Now it may be acceptable, but only your customers can determine how long is "too long."
Friday, February 17, 2006
We're definitely talking about large binary files -- not discrete bits of app data bundled into a huge hairball.

I've come across the notion of asynchronous web services, which seems more reasonable, given the processing time that we can expect before a response is ready.

Friday, February 17, 2006

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz