The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Result set in a distributed system

I am working with a distributed system. What that means for me at the moment is that information is stored in different databases that dont know about each other (so joins across tables is not always possible). In one module, i need to query different modules and then combine the results that are then displayed to the user.
All this brings a lot of processing that could have otherwise been done in SQL into the java module. for example, instead of firing a join in SQL, the module gets results from various sources and then merges them using the Collections classes.
In case of sorting, there are different comparator classes depending on whether the user wants to sort on last modified or created and after sorting the huge result set, the user is presented with the first 25 rows or whatever.

This seems to be a generic problem.

Anyone of you faced this before?
wannabe boy
Monday, May 23, 2005
 
 
If you're going to sort your set and show the first 25 ones (for instance), you can return 25 sorted elements from each original result set -- instead of sorting 10x10000 elements to show only 25 of them.

Of course, if your merge is more complicated than a simple sort, that won't work. In this case ask yourself if approximate results are acceptable.
Pakter
Monday, May 23, 2005
 
 
> In this case ask yourself if approximate results are acceptable.

If not you probably want go the data warehouse route and make a server that presents the data in a unified format rather than merging the results in your own code. You have to figure out how to load the warehouse of course, but that simplifies all other apps built on top of it.
son of parnas
Monday, May 23, 2005
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz