The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Store code in database

Does anyone of an option to store bits of code in a field in a database and execute it at run time? This would be a very neat option to customize our application accoring to our clients' demands.

Thing is we are running on VB6 (now) and will migrate to C#(later some time).

I know I can do this in Access VBA for instance, but in VB, too?
Jack Murdock
Thursday, June 21, 2007
You're treading on very thin water here. Can explain in more detail what you're trying to accomplish exactly?
jonathan Send private email
Thursday, June 21, 2007
Perl is very good at taking 'code' into a 'string' and then 'executing' the code.

In general, this form of "self-modifying code" has been frowned upon -- Perl is technically a "scripting language" and the developer did this as a feature.

The reason it's frowned upon is because it can become a maintenance nightmare.  If you have code in a database, how do you test the code?  How do you insure what came out of the database (to be executed) hasn't been tweaked by a hacker? If the *USER* can customize code in the database, how do you insure he doesn't break the system?  How do you insure the User code is 'good' code?

And does it depend WHEN and WHERE the code in the database is extracted and used?

And if/when you upgrade your program, does the code in the database ALSO get upgraded?  How do you control that process?
Thursday, June 21, 2007
Various large enterprisey apps do this. PeopleSoft has or had tables of SQL statements, PeopleCode, and Application Engine code. They also had a large QA department dedicated to getting it all right, I'm sure.
George Jansen Send private email
Thursday, June 21, 2007
I am aware of some of the disadvantages. To elaborate on my issue.

We sell a B2B application. Nearly all of our clients ask for cusomized fields in our system. The approach we took so far was to add these fields to the standard, so every client would have them available.

This of course leads to overloaded forms beceause each client only needs a subset of the fields. It would not be too difficult to store the field definitions in a database and then show/hide them depending on the client's settings.

But some custom fields are supposed to have funtionality behind them. For example an entry in field A causes an order to be recalculated. But only for one client; not for all.

So what approach would be the best to take beseides:

If "client" = X And "field A" > 0 Then
End If
Jack Murdock
Thursday, June 21, 2007
My company's primary app does this a fair amount.  We sell into a number of different verticals with no customised code.

To do this, we use tables to define user-customizable fields - we offer more than a dozen customizable transaction keys and unlimited non-transactional user-defined fields.  These tables, among other things, define availability, user-changeable labels, pre- or post-add function names and pre- or post-modify function names.

In the OP's example case, the program checks to see if there is a post-modify function defined and if there is, calls the function.  RecaculateOrder seems like something that most customers would want, if not triggered by the same conditions, so it might be a standard system function.  For customers who really want customised processing (few do), we have standardised code that can call SPs, decoding the function name to get the SP name.

I've seen similar setups where programs decode these function names through code tables.  Changing system behavior there is as simple as editing the code table.
a former big-fiver Send private email
Thursday, June 21, 2007
Uh, are we talking about stored procedures?
G Rowland
Thursday, June 21, 2007
No. I mentioned them as one way to get highly customised behavior from non-cusomised application code. Please don't devolve this into a SP-or-not flame war.
a former big-fiver Send private email
Thursday, June 21, 2007
a former big-fiver Send private email
Thursday, June 21, 2007
Search for "declarative" or "data driven" or even "table driven" programming for more on this kind of programming. (But skip any references to "table oriented") Many times the more you put in data and the less you put in code the better so this is a worthy area of study even for single customer products.

I'd avoid any test "if customer x then ..." and instead test "if behavior x desired then ..."  You can map common behavior to many customers and avoid code churn when you add a new customer.

If you can stand coding the solution for each customer, a plug-in architecture like OSGi for Java might be interesting.
Stan James Send private email
Thursday, June 21, 2007
Verticle Tables anyone?

Seems like an easier way to solve the problem then putting code in the DB...(which sounds like a security nightmare)
Vincent Send private email
Thursday, June 21, 2007
There are a number of embeddable scripting languages that you can incorporate into your application which would then let you pull a script out of a text field in the database and execute (or evaluate) it. Most of these languages have an API that allows the host application to define functions and publish variables to the scripts.

TCL, Lua, and Ch are three options that I'm familiar with, but I am sure that there many are others. I seem to recall that there was a solution for Java that could convert scripts into byte code on the fly. I'd bet you could find a similar tool for C#.NET.
Jeffrey Dutky Send private email
Thursday, June 21, 2007
If you wait until you go to C#, you can embed IronPython.  I can't think of ANY solutions that would be easy to migrate from VB6 to .NET.  Also, it's probably better to treat the scripts as seperate plugins off in a different 'tier' than the db itself.  You'll also need to come up with a restricted execution environment, so you can't issue something like System.File.Delete("c:\").
Grant Send private email
Thursday, June 21, 2007
I wrote a dynamic ETL system that read from a metadata repository. Worked like a charm.

Depends on the app, but I love dynamic code (I don't worry about security issues in my apps, they're internal).
Steve Hirsch Send private email
Thursday, June 21, 2007
Uh, my first concerns would be: what kind of cade could be stored in the bd and who can put the code there? These concerns are related.

If the code could be somohow "sandboxed", so that it could ONLY modify the behavior of the current form or transaction, then MAYBE this could be tolerated. BUT, it the code could for instance read some critical information and send it by email, well, then any malicious programmer (and beleive me, they exists, I know this quite well) could introduce spyware in your application. Also considere that as it is stored in a db, then it could be more difficult to audit following standard auditing procedures.
Pablo Chacin Send private email
Friday, June 22, 2007
Designing a framework that can be customized to future, as yet undefined requirements is an extremely challenging task.  I think it would be far more productive merely to adapt the program as necessary when the new requirements come in.  I would avoid the "customized code in a database" approach as I feel the reality will be far more difficult to manage and maintain than simply extending and customizing the application as needed.
Wayne M.
Friday, June 22, 2007
In my experience, moving logic from one place to another does not change the skill required to maintain it. The security issues are real, though.

I'd consider doing it like this (simplifed version): each field on a form is programmed to have a range of behaviors, from being invisible to having multiple methods for different events. As a form is opened, you run through a configuration file that changes the behavior to suit (if different from the default). This may mean just setting constants, but it could include passing functions, if your language permits it.
Peter Vanderwaart
Friday, June 22, 2007
With COM objects, I did once design a pluggable architecture which simply held the ProgIDs and method names in the database.

You can do something similar with reflection when you move to .NET.
Cade Roux Send private email
Friday, June 22, 2007
We currently do something like this for a forms processing system. However, we limit the code to simplified javascript and don't allow the customer to fiddle with it. This code is the output of a code generating tool. Some forms can have up to 1k fields with govt defined formatting, formulas and calculations.

Next generation is going to have to allow the customers to insert their own rules, and the big cheese wants it structured like the outlook rule wizard. I suspect it is partly due to the utter fragility and brittleness of the current in-house tool. I'm currently researching ways to handle calculations/rules on the fly. Probably something in the Jess/CLIPS/Drools/Lua ballpark.

Pablo introduces a concern several of us have seen in the past, and the remedy - logging and audit trails - seem to rub the folks in charge (where I am at) the wrong way.
Peter Send private email
Friday, June 22, 2007
I advise against it. Storing code in database is not a good way to implement customization, simply because it is a great way to sink the program's integrity down the drain. Especially if it is code that is, potentially, written by the customer. The code interacts so tightly in the program that you rarely can afford free mixing; you need to structurize what can be changed before you allow change, or you'll eventually get a hopeless mess that will make your maintenance costs eat up the profits. I have seen it multiple times, often in B2B scenarios with per-client customization.

However, trying to incorporate everything that each of your client needs in the single version can also lead to the same sort of unmaintainable mess, only perhaps slower.

What to do: Since you work for clients who require versatile customization, you have a good base for spotting the patterns in what they need. These patterns can be converted in configurable pieces of code in your application; some of them in the main code base, and some (used more rarely by fewer clients) concentrated in plugin code.

The configuration can be stored in database if you like; it serves as the safe set of alterations that you can allow. Same for the plugin interface; the interface serves as a firewall, protecting your code from runaway side effects and organizing the mess into maintainable regularity. Often you can even spot some pattern in the main code base and encapsulate it in a plugin - when called for, this inevitably results in increased versatility and power on the whole, because you have to think and provide for all cases with a plugin and that has high organizing power.
Sunday, June 24, 2007
I also think that you should treat differently the modifications to user interfaces (which generally accounts for the more of user costumizations) from the modification in actual business logic.

for user interfaces, generally adding events along the processing of requests and allowing custom event handlers will give A LOT of flexibility.

For business logic, the problem is a bit more complex. Designing an application as a framework will lead to application that are easier to modify as you will already consider patterns to introduce additional or modified behaviors, but will not solve the problem of how to introduce this changes, because most of the patterns require you to introduce new classes.

In some CRM products I've seen, they solve this by creating a script or workflow for each business transaction, which can be modified by the customer. These workflows can only access the business objects of the current transaction, so are safer. In some cases, the scripts behing the workflow can ONLY be generated by a tool that prevents the introduction of malicious code.

You could then consider some workflow/rules engines and use them as the drivers of your application logic. One nice think about this engines is that they generaly save the logic in the form of xml documents, not code, which is safer.

Finally, been realistic, not any modification will be possible using this approach, but at some point it would be better to refactor the application and include user modifications in the desing instead of keep adding extensions.
Pablo Chacin Send private email
Monday, June 25, 2007

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz