The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Data exchange between server & app

What is the best way to exchange data between the client apps and the server? XML? JSON? REST?

The client app is custom (i.e. not a browser), but it could use whatever method the webapps use these days.

I am rather new to webapp technologies. If you could also suggest any books that cover your answer (which you personally found to be useful), I would appreciate it.
coder
Monday, December 10, 2007
 
 
What platforms are your client and server running on, and do you want to keep the Data exchange platform neutral?
Ted Elliott
Monday, December 10, 2007
 
 
clients -> MSWin
servers -> LAMP

Platform neutral sounds like a good thing. :)
coder
Monday, December 10, 2007
 
 
I've been very pleased with my first attempt at REST style APIs. What really appeals to me is that I can give consumers a standard link instead of explaining a complex remote procedure call.

JSON is compact and very slick for JavaScript consumption, but XML is a lot easier to debug visually. The "best" answer will of course depend on your specific requirements.
Stan James Send private email
Monday, December 10, 2007
 
 
I would say you'd probably have the best support using xml, i.e. there are plenty of frameworks and libraries out there for parsing and reading xml.  If you're sending lots of data or need very fast communication you might want to try JSON.  If you're using .NET on your client, I maintain an open-source JSON serializer library for C# on google code: http://jsonexserializer.googlecode.com .  You can find others for a wide variety of languages at http://www.json.org.
Ted Elliott
Monday, December 10, 2007
 
 
What are some of the security issues with data exchange using JSON and XML? Isn't this similar to the "SQL injection attack" scenarios?
coder
Monday, December 10, 2007
 
 
I'd like to slightly diverge from the above question and ask a more general one...

The problem I see with all of today's techniques (some of which were mentioned above) is that they lack static typing to ensure that a simple typo does not break your program.

One approach I am considering is having Java on both ends, sharing a class and allowing clients to invoke server-side methods off that class using a RMI-like protocol (i.e. RPC). The only downside I see to this approach is that it requires Java on both ends of the wire (but this isn't a problem for my particular product). What are your thoughts on this and why did the industry move away from this kind of approach?
Gili Send private email
Monday, December 10, 2007
 
 
Having recently developed a Java RMI app, I can say there is a big difference "RMI like" and "RMI". Java RMI is not a complete solution, especially for apps that cross the internet(NAT and firewalls in particular), and we've had issues with computers that have two network cards in the apps I'm involved with. All those problems are surmountable.

As to your desire to have "type checking" of some sort between client and server, I totally understand. There are several ways to go about. A common approach(that I use, and several frameworks, including Wasabi and GWT, use) is code generation.

There are things like SOAP and Web Services that deal with some of that too, but they are not always pleasant to work with, in my experience(honestly, I prefer raw HTTP to those, however, for the appropriate hourly rate, all technologies can be pleasant to work with).
Matt Send private email
Monday, December 10, 2007
 
 
Great question for Design of Software forum by the way.
Li-fan Chen Send private email
Monday, December 10, 2007
 
 
+1 Stan James

I've used SOAP before, and whilst it sounds nice in principle, in practice I found it cumbersome.  I'm currently using a REST approach and enjoy the simplicity of it.

This article does a good job of explaining the differences between REST and SOAP: http://bitworking.org/news/125/REST-and-WS

These two are good designing a client/server API using REST: http://pragdave.pragprog.com/pragdave/2007/03/the_radar_archi.html
http://bitworking.org/news/How_to_create_a_REST_Protocol

HTH
Dr Bunsen
Tuesday, December 11, 2007
 
 
An important aspect that has been neglected in this discussion is the export medium: to you plan to use HTTP, TCP/IP or something else (e.g. message queue)? REST obviously means HTTP, but XML/SOAP can be transported over anything. In this context: do you need guaranteed delivery (transactions)? Server-initiated messages or only client-initiated messages? Do you need to go through firewalls/proxies? What is the expected volume of the data being exchanged? Any special security/authentication requirements? Without knowing much more about your requirements it's very difficult to propose a valid solution.
Dan Shappir Send private email
Tuesday, December 11, 2007
 
 
On typing: It's possible to describe a syntactical schema for XML, using technologies such as XSD or similar. I don't see any reason to rely on a platform specific protocol, such as RMI.
Troels Knak-Nielsen Send private email
Tuesday, December 11, 2007
 
 
In situations where I have control over both the server implementation and the client implementation (either directly or via a shipped library), I favour a REST style interface, with either custom or off-the-shelf data serialization, generally using XML. (Investigate XML stream parsers if doing your own serialization of your data)

For situations where you are exposing a general service to a varying set of consumers, I would investigate the complexities of SOAP or other 'standard' web service mechanisms...unless, as above, you can provide them a good client interface library to use.
Dan Fleet Send private email
Tuesday, December 11, 2007
 
 
We use ICE (http://www.zeroc.com) to talk between a webapp and a backend service.  Its free for users who aren't making money from the software (though you should review the license agreement to make sure its free for you).  ICE is very high quality software.

Cheers,
Andrew Bell Send private email
Tuesday, December 11, 2007
 
 
We're using XML with web services. But then we have apps ranging from VB3/4 to .NET 2.0.

We're looking into JSON for one web-based application because the XML being pushed back and forth can sometimes be large enough to cause timeouts. Which is somewhat odd because the form that can cause these problems is getting phased out by regulatory changes next year.
Peter Send private email
Tuesday, December 11, 2007
 
 
I've got a Java-oriented question but it should apply equally to XML, JSON, REST, etc.

The way I see it, many developers end up choosing an exchange protocol but then proceed to transfer important data outside of it. For example, Azureus uses the BitTorrent protocol which is platform independent but then proceeds to transfer plugins in a less portable format. The way I see it, if you're going to be discussing a *good* data exchange protocol it should really encompass *all* your data exchanges because you're only as good as your weakest link.

For example... I've never really understood when one should be using RMI(-like) versus RMI-IIOP versus the more modern Web Services.

I've posted a similar question here: http://forum.java.sun.com/thread.jspa?threadID=5245926

My understanding is that:

RMI: Fastest, easiest to use, most flexible but the least portable (requires Java on both ends)

RMI-IIOP: Slower, more difficult to use, doesn't support "mobile code" but most applications don't use it anyway. This is slightly more portable because there is a RMI-IIOP .NET client.

Web services: Slowest, most difficult to use (you lose static typing), least flexible but the most portable.


I think "mobile code" is very important because of the Azureus example I mentioned above. When you think about it, both Azureus and FireFox make use of mobile code when they downloads plugins and execute them locally. Java mobile code might be less portable (across languages) but it is far more seamless than other solutions.

Any thoughts on this?
Gili Send private email
Wednesday, December 12, 2007
 
 
>Web services: Slowest, most difficult to use (you lose
>static typing), least flexible but the most portable.

You only lose it, if that's what you want. There's nothing preventing you from sending type information together with the values (XMLRPC) or to declare an explicit schema (SOAP).

Now, it's true that web services may appear as a bit more labour, compared to RMI, but on the plus side, they tend to be very explicit. This in turn, makes one less likely to do stupid things. Some times, abstractions are more trouble, than a help.
Troels Knak-Nielsen Send private email
Wednesday, December 12, 2007
 
 
What if the data to be exchanged is files such as PDF?
coder
Thursday, December 13, 2007
 
 
Troels,

Do you get compile-time errors if you violate the schema?
Gili Send private email
Thursday, December 13, 2007
 
 
>
Do you get compile-time errors if you violate the schema?

No, but neither would you with RMI; That would be a run-time error.
Troels Knak-Nielsen Send private email
Friday, December 14, 2007
 
 
I am expecting to have a schema (with RMI this consists of a Java class) defined and validated at compile-time. This schema defines what is sent between the client and server. With RMI the only time I would have a runtime error as you describes is if the client and server class definitions differ from one another. I assume that in web services your schema is defined in some XML file and that you would not get a compile-time error even if you you violate the local view of the schema. Maybe you meant that they have a post-compilation step that verifies the code does not violate the schema?

My second point is that the data exchange mechanism should be all-encompassing such that nothing is sent out-of-bounds. That is, if I am using web services to gain platform/language portability but then my application ships Windows DLLs across the wire then my portability is actually limited by that DLL and I might as well have used a Windows-specific data exchange mechanism to begin with. That is, I am only as portable as my weakest link and web services is only justified if *all* my data is truly platform/language independent. I suspect quite a few applications violate this rule.
Gili Send private email
Friday, December 14, 2007
 
 
What should I use if the content to be exchanged is big in size such as PDF files? FTP?
coder
Saturday, December 15, 2007
 
 
> What should I use if the content to be exchanged is
> big in size such as PDF files? FTP?

You can transfer files over HTTP. Once they get over a couple of megabytes, it's perhaps worth to investigate other means. That depends a lot on the intended audience though. If the uploader can be expected to have good network connection, I would be confident until something like 50 megabytes in size or such (Completely arbitrary number -- Don't put too much into it). Beyond that, I would probably look to FTP or SCP (If security is an issue).
Troels Knak-Nielsen Send private email
Monday, December 17, 2007
 
 
> Maybe you meant that they have a post-compilation step
> that verifies the code does not violate the schema?

Yes. There are frameworks for this, or you could roll your own.
But since you need to take hand on run time exceptions anyway (Networks are inherently unreliable), you might as well handle violation of protocol the same way.
SOAP isn't much different from RMI, except that it uses XML as transport layer and is designed to be language agnostic. XMLRPC and REST based services are much looser designs, but you should keep in mind, that while you see a strict protocol as hepful, it is just bureaucracy for other platforms. The choice between these options does therefore rely to a great extend on the intended audience.

> My second point is that the data exchange mechanism
> should be all-encompassing such that nothing is sent
> out-of-bounds.

That's exactly the reason why you should be stuffing everything through the platform independent interface, that HTTP is. I like the quote "The Internet is the New Unix" --  http://shiflett.org/blog/2007/oct/the-internet-is-the-new-unix
Troels Knak-Nielsen Send private email
Monday, December 17, 2007
 
 
>> My second point is that the data exchange mechanism
>> should be all-encompassing such that nothing is sent
>> out-of-bounds.

> That's exactly the reason why you should be stuffing
> everything through the platform independent interface, that
> HTTP is. I like the quote "The Internet is the New Unix" --
> http://shiflett.org/blog/2007/oct/the-internet-is-the-new-unix

Fair enough, but my point was that if you're transferring native libraries then it doesn't really matter what transport you use because your weakest link are those libraries. That is, if you're transferring C# libraries then you might as well use a C#-based protocol because both ends have to have CLR anyway. The same goes for if you're using Java.

I think that platform agnostic simply means "browser platform" which in turn means HTTP/JS/XML, which is a platform in its own right. I think this only makes sense for browser-based applications that do not involve external downloads.
Gili Send private email
Tuesday, December 18, 2007
 
 
> ... if you're transferring native libraries then it
> doesn't really matter what transport you use because your
> weakest link are those libraries. That is, if you're
> transferring C# libraries then you might as well use a
> C#-based protocol because both ends have to have CLR
> anyway. The same goes for if you're using Java.

I think I understand you now; You mean to transfer binary code between client and server and then let the end point execute the received binary? That's a peculiar protocol to use -- Why would you need that?
Troels Knak-Nielsen Send private email
Tuesday, December 18, 2007
 
 
> I think I understand you now; You mean to transfer binary
> code between client and server and then let the end point
> execute the received binary? That's a peculiar protocol to
> use -- Why would you need that?

Pretty much any plugin system works this way (though maybe that's not so obvious). Servers stream plugins to clients which install these (think FireFox, Azureus, etc). This is an example of the server sending executable code to the client.

A much rarer use-case is when the client sends code to the server to execute. Most protocols work by specifying a limit subset of terms that clients may search over when looking for files, plugins, etc. An alternate protocol is to let the client send code to the server which could search over the pool of data in any way it sees fit. In a P2P use-case you could conceive the following search: "Find me a torrent whole contents contains a file with a hash of X" or "Find me a torrent containing a compressed ZIP file containing filename X". You can't currently do these things because servers don't accept executable code from clients. There is an obvious security danger there but there are ways to protect yourself, especially using Java SecurityManager, signed objects, etc.
Gili Tzabari Send private email
Thursday, December 20, 2007
 
 
I can see the idea of sending executable code, but I think that's something very different from a service. Or at least it's a more liberal interpretation of the concept, than I would use.
Troels Knak-Nielsen Send private email
Saturday, December 22, 2007
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz