The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Complex Unit testing

I'm trying to get on board the unit testing bandwagon. I realize that it's a very useful tool for improving productivity and software quality. I understand the concept for methods with well-defined data values and inputs and outputs. However, I'm suffering from some conceptual road blocks with more complex tests.

Some examples:
1. User interfaces.
2. Complex user interfaces. Our s/w has a map that allows users to place objects on the map.
3. File creation/manipulation. Our s/w imports data by unpacking archives (zip, tar).
4. Database stuff. The map is tied to a backend database. When objects are created on the map, database is updated. Some imported data is stored in the database.

Any pointers to information on unit testing these kinds of things?

Developer #13
Wednesday, September 28, 2005

"Some examples:
1. User interfaces.
2. Complex user interfaces. Our s/w has a map that allows users to place objects on the map.
3. File creation/manipulation. Our s/w imports data by unpacking archives (zip, tar).
4. Database stuff. The map is tied to a backend database. When objects are created on the map, database is updated. Some imported data is stored in the database."

Tricky. I have had similar problems very very often.

Here's how I solved them - I would love to discuss this stuff  further :-)

1) Personally I find I'm not actually wanting to test the UI, rather I was wanting to test the logic that was intertwined with the UI code. Often much of the 'feature driven' logic is too embedded in the UI code...

I found I abstract much of the code out of the UI into their own classes/functions that were testable and provide the correct contract to the (now thinner) UI code.

I could then prove the functionality that the UI was trying to use and cut down grief for myself when building the UI (actually testing this in itself is another problem that requires other tools).

2) Not sure I understand this one exactly - however, if the abstractions are right then anything is testable. Perhaps by pulling logic out of the map you can test + make the map thinner - the 'business logic' is then proven , you know it works, all you have to do is then deal with its contract to your map.

3) I've had similar problems - abstraction again. If I can test what goes in and what comes out ( i.e. I know what goes in I *must* know what comes out ) then the abstraction is more correct - you might have to create (what I call) 'control' files and data.

4) Abstraction - you might be able to 'mock' the data. If not, then perhaps drive more 'integration' type tests from the front end (had this problem too). For example, if you can add an object to the map and you can get an object from the map then perhaps you can write an 'integration' test to add a known object, then get it, then compare it with what you expect - thus proving the mechanism and data storage works. This isn't really unit testing perse, BUT, it offers and end to end integration test, which if you maintain and execute often can also keep the integrety of your application in place and can complement proper unit tests. I use JUnit framework (modification of) for end-to-end tests when I need them.

Hope this helps - just my 2 cents, and may not help, but I have found working in this way gives a high quality product, gives me a better abstraction with little overhead, and makes testing a lot easier. "If its difficult to test, perhaps its wrong??"

GK Send private email
Wednesday, September 28, 2005
The current MSDN Magazine has an article about "User Interface Process (UIP) Application Block – Version 2.0" which supposedly provides a framework for putting most of the logic of a user interface into configuration files and classes that are subject to unit testing. I wonder if anyone has had success with that.

For testing "database stuff" we have had good luck with the concept of a Test database that never changes (except for schema updates). Each unit test assumes that the Test database is in its pristine form, and leaves it that way by cleaning up after itself. It would also be possible to implement tests that clean up by restoring the database from its backup file. The only trick is that when a test fails, it may corrupt the Test database and cause other tests to fail, in which case it may be tedious to find out which test is the culprit.
Howard Send private email
Wednesday, September 28, 2005
Read this:

and the links it points to for a pattern for taking logic out of the UI code.  I've been using it with some success on my project.  The sticky part comes when the interactions get very complex -- hard to imagine sending information to a class and have it send it back to you instead of just updating some fields yourself -- I use it to make sure my UI class has extremely simple methods (very little beyond the normal UI API stuff).
Lou Franco Send private email
Wednesday, September 28, 2005
I experience similar problems in much of the work I do, in my ai research code.

I often find myself writing code that consumes tens of thousands of files as input and produces some sort of statistical index file as output. Writing a unit test for that kind of code is difficult.

Even more difficult is when I write code that has to consume one or more statistical indices along with an input message and then produce some sort of output (a categorical classification, a new node in a decision tree, a transformation on the original message, etc) based on the results of analysing the message against the statistical index.

I've tried to write unit tests for code like that, but I'm completely stumped.

My conslusion: unit testing is not the answer for everything. In many cases, regression testing is the right answer. In other cases, functional tests or acceptance tests are more appropriate. But some classes just cannot be validated as correct by using unit tests.
BenjiSmith Send private email
Wednesday, September 28, 2005
I've heard people say that Fit and Fitnesse are good tools to use for UI testing, but I've never spent enough time to really grok how they work:

This is a form of acceptance testing,  different from unit testing or regression testing.
Herbert Sitz Send private email
Wednesday, September 28, 2005
My NZD$0.02...

It may sound like nit-picking but the four points listed are not really for unit tests. They're more integration tests because you are interfacing with third-party componants (database/file system).  In more detail:

1 & 2) The GUI should be seperate from the code that makes the GUI do something you want to test (e.g - you should be able to call the routine that places the particular object on the map at a particular point) by code, not just by having someone move a mouse and click.  As mentioned, FIT ( ) and Fitnesse can help you here.  I've found FIT useful for helping me design code that clearly seperates UI and logic.  So don't write unit tests but use FIT to do some integration tests.

3) Similar to above.  I'm assuming you want to test the code that imports and unpacks files.  The routine to test should not be dependant on the user specifying the file, instead the file name or pointer to an open file should be passed in - that way you can test with good files, corrupt files, etc..

4) Databases - there's a reason why unit tests use mock objects.  It keeps the tests simple, quick and they test only the logic, not the integration with the database.  Use mock objects.  If you want to test the integration, I suggest have a database specifically for integration tests  to connect to.  That way you can populate the database with all sorts of test data and can determine if it was written correctly (assuming it's deterministic data).

Hope that points you somewhere useful
Paul Norrie Send private email
Wednesday, September 28, 2005
Great replies everyone. Thanks! I'll do some research on the topics mentioned. I've got a lot to learn, so I'll probably be back with follow-up questions in the future.
Developer #13
Wednesday, September 28, 2005
When you write code to "read in a file", methods needn't take in a file. They can take an istream (C++) or an InputStream (java). And in the actual application you can use a file stream (ifstream in C++ / FileInputStream in Java), and in the unit test you can use a string stream (istringstream in C++ / StringInputStream in Java).

There are output stream counterparts for these. Also, please forgive me if I made any mistakes. I don't want to get out the reference right now.

If you're doing a binary file, you may want to write a quick perl script to encode it in your source code in a way that is good for a text file.

So, there are some ideas for files. For user interfaces, nothing for regular user interfaces, and for complex widgets, such as maps, get good abstraction so you can test most of the code. And for databases, a good test database setup is good, but it won't do everything. Databases are tricky.
Ben Atkin Send private email
Thursday, September 29, 2005
Regarding the database part of testing, this site is a good spot to start:
Herbert Sitz Send private email
Thursday, September 29, 2005
Also, don't assume that all tests should be automated (although the agile approach relies heavily on the idea that an automated unit test suite will find all regressions due to refactoring.)  I've found Brian Marick's paper on "When Should a Test be Automated?" well worth considering:
DaveW Send private email
Tuesday, October 18, 2005

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz