The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

soft-real time, data/gui intensive applications design

I'd like to know how people approach the design of software that deals with a message bus (tibco rv, mq, jms, simple network) and has lots of data that needs to be managed.

Specifically, I'm thinking of applications for the financial industry where a large number of trades, orders, quotes might be flowing in over the network.  Even more specifically, I'm thinking of how to manage a graphic window that may contain this data (think of trade blotters).  It is one thing to simply populate a table (gridview, jtable) as the data comes in, but things get a little more complicated when I need to provide functionality such as:
- changing the color of a cell based on pre-defined criteria
- allowing users to define their own criteria for the color of cell containing data
- filtering on some 'key' (but not deleting the data that's filtered out since it may be 'unfiltered')

I have built such apps but they seem very ad-hoc.  I certainly make use of producer/consumer, observer pattern, etc., but this is a common enough problem that some interesting solutions should exist for it.

Basically I'm looking for a general idea of how people approach this problem.  If there are books, patterns or any other resources, please point them out.  Thanks!
Falcon Send private email
Thursday, October 27, 2005
I have used the virtual ui approach in the past where the ui is just responsible for painting the strings based on the parameters it is told. A mediator in placed in between the data message and the ui to govern what rules apply and how the ui should do the painting. So in case of the grid or jtable, it is just busy painting strings into the cell and each cell gets a parameter on what and how to paint itself.
Other classes are responsible for figuring out what the filter on the data is, and provide a view on just the specific filtered data(so for example having a data table and multiple views on the data).
Amit Malhotra Send private email
Thursday, October 27, 2005
The approach that I would recommend is to find an abstraction that already works, and try to leverage that as much as possible.  The simplest approach for most GUI applications is to leverage the relational model, and the various higher-level bits that hang off that (e.g., ADO.Net).

For real-time apps for Wall St., you have an addl problem, as the data is changing all the time.  You can't afford to query a DBMS 100 times per second, and most of the data isn't even in a DBMS, anyway.

However, that doesn't mean you can't use one of the higher-level abstractions mentioned earlier.  For instance, you can take market data that is flowing over a message bus, and stuff in an ADO.Net DataTable.  You can then bind that object to all sorts of GUI widgets that understand how to display the data.

Which brings up the most important point -- you need to choose a higher-level abstraction that provides a "notification interface" that will tell the GUI to re-draw itself when (some of) the underlying data changes.  ADO and ADO.Net have a pretty good notification interface that is understood by lots of widgets.  Java's Rowset has a notification interface as well, but the data-binding support in Java is not nearly as mature as it is in .Net.

(Shameless plug follows...)

FWIW, my company develops and markets software to do just what you want.  You can check it out at, or drop me an email if you're interested in more information.
BillT Send private email
Thursday, October 27, 2005
The problem with databinding in .Net is that once the traffic of inflowing messages increases, repaint of gui becomes increasingly slower since the repaint is not selective on what part it repaints.

Take a look at this virtual grid article in the .Net space.

By using such 3rd party ui components, you can concentrate on  implementing business rules based on data and user preferences.

Btw, I wouldn't recommend hitting(polling) the database, rather an event based model.
Amit Malhotra Send private email
Thursday, October 27, 2005
I once made an 'event graph' component, a bit like the RAM usage type of graphs you have in the Windows Task Manager (mine, stereotypically, was share-price-tracking).

These kind of components have many similiar goals to a textual display of the same kind of event processing:

* there are lots of small individual datapoints, and the data has to be processed as quickly as possible as old information is worse than useless - it is actually misleading.

* performance is critical; drawing is surprisingly slow, and the display server has a lot of housekeeping to do that you are probably not aware of.

Here are the some of the design decisions I made in my bespoke implementation:

* scrolling (text consoles, graphs etc) are a common graphics problem and many graphics frameworks.  The platform I was using had a 'copy_rect' function which was like a bit_blt except that it was copying the framebuffer across.  This was because the window was guarenteed on-top, but I'd anticipate that you'll find similiar shortcuts on other platforms.

* there is a predefined amount of screenspace to be filled, so only the most recent datapoints need plotting.

* the maths is far easier if the data-points have a uniform row/column size

* the screen and the human are surprisingly slow to update; there is no point animating as fast as you can eat the stats, only as fast as the user can perceive it.  Animations usually max out at about 15 fps, and limiting it frees up CPU for crunching the stats

* it is often neat to have one thread for eating the stats and keeping track of the most recent n points - the 'window'; and another to be polling this window to generate the display n times per second.  Often the drawing thread is lower priority, so you don't delay datapoints

Depending upon your scenerio and target hardware (we had a whole screenful of small graph thumbnails, so whilst each individually was relatively lightweight work, it all added together to really stretch the 386) this might be overengineering or otherwise.

.NET and stuff?  Repopulating listboxes or grids or whatnot?  Sounds like you have loads of crunch and relatively few or slow datapoints.

Just the algorithms behind many listbox and grid models is against you.  They use an array, which means to display a top n items you have to keep shuffling down the items and inserting at the top and removing from the bottom etc.  Historic displays are best solved with (cicular) queues instead..
new nick, new rep
Friday, October 28, 2005
re: "The problem with databinding in .Net is that once the traffic of inflowing messages increases, repaint of gui becomes increasingly slower since the repaint is not selective on what part it repaints."

Right, and there's also the issue of humans needing to be able to see the data.  Even if the GUI could do 10,000 repaints per sec., a user could never follow that.

Our software allows for that with an option to specify the "notifyInterval".  With notifyInterval set to 500ms, all updates are buffered internally and presented to the GUI every .5 sec.   

Also, most of the MS widgets (e.g., DataGrid) appear to be somewhat selecective about what they update on the GUI.  This is possible because the notify mechanism (IBindingList:ListChanged) has arguments to specify what changed.
BillT Send private email
Friday, October 28, 2005

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz