The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Configuration store

Hello,

Let's assume that we have a large, hierarchical and heterogeneous configuration for a real-world software product. This configuration has to be stored and retrieved from the disk.

What would be the best OO approach for this?

I came up with multiple solutions, but none of these seem to be elegant or pratically working.

Problems that we have to deal with:
- Configuration structure changes
- Full compatibility with previous versions
- Human-reable, hierarchical storage

Approach 1) "One Ring" One large class which knows how to read and write the configuration file. The advantage is that everything is at one place, I have full control over the configuration. Unfortunately, this class soon becomes bloated, simply due to the size of the configuration.

Approach 2) "Save yourself" Configuration classes support their own persistency. The problem with this approach is that the configuration might be restructured as the software evolves and depending on how automatic the persistency is, it may cause troubles. Moverover, classes become bloated by their own persistency code and also they publish information that they should not have (?).

Approach 3) "Interpret" Turn each configuration option into a command which is interpreted when the configuration is loaded. This does very well with compatibility, but the storage becomes flat so it is not easy to read anymore. Also smells a little bloated.

Thoughts?
pb
Monday, May 30, 2005
 
 
(3) has served me well in the past, using Python as the configuration language. The config file was a python file, which the C code executed, and then used whatever values that Python code had set.

The power this gives you is not easy to grasp until you need it - e.g. your python script can query a database to get the values, rather than use a local file; Or, it could try to guess reasonable defaults; Or .... whatever.

Do note, however, that the result often isn't editable by a program, unless it was previously written by a program.

I highly recommend (3) with a mature scripting language (Python, Lua, TCL, Io, whatever).
Ori Berger
Monday, May 30, 2005
 
 
Since we do about 95% database work, we store our configuration in the database. There's only one configuration option that's not in the database -- and that's the pointer to the database. All other config information is stored in the DB.

Physically, according to your terminology it's "flat" -- a single table. Logically, it's a heirarchy. It's implemented where the config is named (and used) much like the Windows Regristry. The configuration values are simply key/value pairs.For example, we might have:

\\USER\SomeUser\Defaults\PreferredFont\FontName      Tahoma
\\USER\SomeUser\Defaults\PreferredFont\FontSize      8
\\USER\SomeUser\Defaults\Queries\QueryTimeout        30

\\SYSTEM\Users\Defaults\PreferredFont\FontName      Arial
\\SYSTEM\Users\Defaults\PreferredFont\FontSize      8
\\SYSTEM\Users\Defaults\Queries\QueryTimeout        10
\\SYSTEM\Users\Defaults\Windows\WindowStyle        Maximized

There's 4 basic levels:

\\USER
\\MACHINE
\\APPLICATION
\\SYSTEM

We'll look for a USER level setting, if it's not there, we'll look for the setting on the MACHINE level, if it's not there, we'll look for the setting on the APPLICATION level, if it's not there, we'll look for the setting on the SYSTEM level, and finally, if it's not there, we'll throw an exception.

This allows us to add new config settings at the SYSTEM level, and be assured that those settings will be available to all users. If a user changes a config setting, if they're an admin on their machine, it asks if they want to save the setting for the current user (USER) or for all users of the computer(MACHINE). If they're not an administrator for either the particular application or system, they don't get to touch those (APPLICATION, SYSTEM).

We go with the "One Class" option. There's a ConfigurationManager class that knows how to read/write these values. It is not at all bloated. Basically, this class needs to know a few things about the config settings -- e.g. a font setting needs to know how to display the font dialog and a color setting needs to know how to display a color picker dialog, but these are pretty rare. Most of the options are simply a domain -- either (as in the case of the QueryTimeout option) an integer, a boolean, or a predefined domain list that's stored in another table:

Domain        Sequence  Key        Value
WindowStyle    1          Maximised  1
WindowStyle    2          Minimised  2
WindowStyle    3          Hidden      4
PrintTo        1          Preview    1
PrintTo        2          Printer    2

etc.

So, the ConfigurationManager class needs to know about what domain the individual configuration setting comes from. These are (for us):

(1) Integer (This needs a min/max allowable range)
(2) Long    (This needs a min/max allowable range)
(3) Boolean
(4) String  (This needs a max allowable length)
(5) Binary  (This needs a max allowable length)
(6) Domain (from the above domain table)
(7) Color  (and it uses the Win color chooser)
(8) Font  (and it uses the Win font chooser)
(9) Folder (and it uses the Win folder browser dialog)
(10) File  (and it uses the built in Win File common dlg)

That's about it. Doesn't seem too "bloated" to me and it works well for us.

The only down side to this (for us) is that it's string based -- based on the key. There's no real "type safety" here. There's nothing to stop a programmer from pulling a FontSize configuration and stuffing into, say the FontName for a control. We'll be working on that for the next version, but if you're reasonably careful, you don't run into this too much, and if you do, it's quite easy for a developer to figure out where s/he went wrong.
Sgt.Sausage Send private email
Monday, May 30, 2005
 
 
How about configuration classes that save and load themselves as XML fragments. You register these classes with a master configuration class that parses the configuration at a high level and invokes the appropriate configuration class to read it's section when required. To save, it performs the reverse operation and cycles through each class in turn to get the necessary XML fragment to save in the larger file.

XML is heirarchal and human readable so it meets that criteria. I also find XML to be pretty flexible in terms of handling structure changes and this methodology would allow you to change the broader structure as needed. For the fragments, each configuration class could write the version number in it's fragment and read back appropriately to maintain backwards compatibility.
Gerald
Monday, May 30, 2005
 
 
I would definitely look at using XML and see if it fits. I am a huge fan of XML, so I'm a bit biased I guess.

XML is human readable and editable, it is easy to version information and easy to extend without breaking what's currently in place.
Neville Franks Send private email
Monday, May 30, 2005
 
 
Thanks for your responses. We currently use the "One Ring" approach (one big class which knows how to load and save), which allows us to deal with structure changes well.

The problem is that the configuration consists of 50 classes (yes, large config indeed), so the config persistency handler is about 200Kb and this is just WriteString() and ReadString() basically. It is hard to navigate in a code as large as this and it is not very elegant.

Ori Berger > The "interpreter" approach with script support is very powerful indeed, in fact I think I will use this idea in the future for another task if you do not mind. On Win32, you have a great opportunity for script language integration by the ActiveScript API (ability to host VBScript/JScript/ActivePerl/ActivePython or even PHP in your application).

Sgt.Sausage > I may have been wrong about the use of word "bloated" (sorry, English is not my native language) -- I meant "big and hard to navigate". It appears to be quite hard to avoid making such class big.

Gerald > This sounds promising: it distributes code and also can deal with structure changes. The only issue I see here is the hierarchy.

Neville Franks > I'm not as huge fan as XML as the industry in general, although I believe it may be a perfect choice for some specific tasks. Parsing an 500Kb XML file, however, takes time and memory, and SAX may not be an option for all cases.
pb
Tuesday, May 31, 2005
 
 
If you want it to be more heirarchal, simply allow the configuration classes to be nested and have children. Thus when you register a class you could register it as being under or owned by another configuration class.

As to the overhead of parsing, I'm assuming this is only going to happen once at application startup so it should be a one time hit. If this is running in a J2EE application server, just read the configuration in a startup class and cache it.
Gerald
Tuesday, May 31, 2005
 
 
You don't say if you expect the on disk configuration to be edited by hand or only through a tool?

If you expect the configuration to be edited then XML seems your best bet.

Versioning is the biggest pain in the but. I would combine your methods. I would have one object that represents the whole configuration as a single data source. It's unlikely your configuration maps to real objects in your program so I wouldn't bother making domain objects, though I would use a tool to create XML data encap objects, simply for ease of use. But to the program they would always go through your proxy object. In your proxy you can deal with versioning hopefully transparent to the application, except where the program must actually do different things for different versions. Hopefully you can just structure your program to do the right thing if certain is available or not.
son of parnas
Tuesday, May 31, 2005
 
 
For database applications, I'm a big fan of having *everything* in the database EXCEPT for the connection string.
KC Send private email
Tuesday, May 31, 2005
 
 
Nothing stopping you from storing XML in the database, I've done it many times in the past for heirarchal structures that had no reporting requirement.
Gerald
Tuesday, May 31, 2005
 
 
XML in a database works fine as long as you don't have to search it.  Then it becomes a bit of a nuisance.

If you're just parsing it for transport or display, this works wonders.
KC Send private email
Wednesday, June 01, 2005
 
 
If app.config and/or web.config (in the .NET case) is not good enough to store your configuration, perhaps you need to reconsider what you are storing as configuration information.

Doing gold-plated configurations is one easy way to make your app start to resemble a rube-goldberg device.  If you really need that much configuration (e.g. beyond simple name/value pairs), perhaps what you really need to do is handle the "config" data in the DB schema somehow - making it part of the formal model.
SourAaron Send private email
Wednesday, June 01, 2005
 
 
Gerald > If you want it to be more heirarchal, simply allow the configuration classes to be nested and have children.

Nesting could break the restructuring I think. If we allow the configuration classes to know their location in the configuration class, then maybe we allow them to know too much.

son of parnas > If you expect the configuration to be edited then XML seems your best bet.

Provided that the XML is generated by MSXML (which appears to be a good choice on Win32), then the resulting XML can be hardly edited, because MSXML does not add CRLF's and does not identing. Maybe an XSLT transformation could do the trick, though.
pb
Thursday, June 02, 2005
 
 
Neville Franks > I'm not as huge fan as XML as the industry in general, although I believe it may be a perfect choice for some specific tasks. Parsing an 500Kb XML file, however, takes time and memory, and SAX may not be an option for all cases.

I really don't think time is an issue you. I can load and parse a 10Mb XML file in less than 1 sec on a 550Mhz P3. In fact Surfulater (see sig) demands this level of performance.

And yes the XML files can be hand edited. I can't comment on MSXML though. If you want more information contect me directly.
Neville Franks Send private email
Thursday, June 02, 2005
 
 
I'm re-examining this very issue now.  We used to have the "One Ring" approach but it's gets unwieldly when you have a lot of configuration options that are changed both manually by editing a file and visually in the app. 

Our configuration options generally pertain to customising business rules rather than users choosing pretty fonts they like.  This and the unweidly "One Ring" (aka. monolithic-type, resembles everyones first attempts at C++) method have caused me to look at two options:

1) Your option 3) where we use some script like a modelling language or python to define configurations.

2) Having a master ConfigurationManager that every other class knows about and acts as a mediator to classes that configure specific areas of the app. (similar to option 2)

Note these are not exclusive options.

And on the subject of XML and editing by hand - WHY?  Editing XML by hand is time consuming and error prone.  I think XML is near useless for many things unless you have defined a DTD.  Here endeth the rant.
Paul Norrie Send private email
Friday, June 17, 2005
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz