The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Can I generate all this CRUD stuff?

Hi there,

I was wondering what people use to generate Java forms to Create, Read, Update and Delete database entries? There's got to be a better way then writing all this stupid code by hand.

Thanks in advance
Dreamer
Friday, March 03, 2006
 
 
That is what Visual Studio winform databinding does. For Java world, check with Borland's database components. They might have similar functions
Bo
Friday, March 03, 2006
 
 
Some sort of code generation may be what you're looking for.

Codesmith is a good general purpose code generator.  It's .NET based but can be used to automate code generation in any language:  http://www.codesmithtools.com

If you don't get any better advice then this may be a good place to start looking:
http://www.codegeneration.net/generators-by-language.php?language=4
Herbert Sitz Send private email
Friday, March 03, 2006
 
 
Enterprise Library also helps a bit. Make sure you understand it well (read the source). Don't be in a situation where you create three layers of CRUD (Business Object in csharp, Data Accessors in ADO.NET, complete CRUD set in SProcs) unless you can justify it. What justifies sprocs are business rules you want to run in cursors or sub queries. Data Accessors are great for fending off nulls and friends and unifying/codifying exception handling. Business objects are great once the design has stablized enough to the point where you can start coding for the future (revisiting code in the future is never fun when you have all three tier inside the button handler!). Good luck, let us know what you do.
Li-fan Chen Send private email
Friday, March 03, 2006
 
 
Uggh... code generation. In the app I work on we originally had a whole bunch of "automatically generated" crud screens. We ended up scraping them all. I personally dislike code generation for the following reasons.

1) It typically results is sloppy code. The person doing the generation doesn't have get to grow from the process of creating the screens by hand. This growing process usually results in a better architecture because it exposes the pain points. Conversely, there is no pain in just clicking the "Generate Crap" button so the original implementer doesn't feel the real pain until a year later when maintenance begins.

2) There is no such thing as a standard CRUD app. Every screen has additional business rules and requirements that simply aren't captured through the database schema alone. Automatically generated screens are typically too rigid and don't easily allow for enhancement for business requirements. Changing the generated code in any way causes a maintenace nightmare when someone regenerates the screens again and loses the customization.

3) The original code geneartion scripts are often lost over time. They also become a maintenance headache all their own. Someone has to be in charge of creating and maintaining them as well as educating others on their use. Home grown code generation is usually the worst. Especially when the guy who created all of the original scripts (and the scripting engine) leaves the company.

4) Screens created off of database tables are completely "data-centric". They aren't intuitive to the user. You end up with screens like "Customer Master Maintenance" and "Customer Detail Maintenance". Users don't understand them because they are derived from a data model that doesn't really model a user's way of thinking. Column names are often shown as headings and they don't have real world equivalents so users are stuck wondering how to use the system and what to type into each column. This is a common problem with products that are designed entirely from a developer's point of view. For more on this, see some of the articles that Microsoft has published on Inductive User Interfaces.

Don't get me wrong. Code generation can work if used in conjunction with solid design principles. But it shouldn't be used as a substitute for creating a solid architecture. Code generation can be easily abused and it often does more harm than good. After all, a good OO design results in very little (if any) duplicated code. If nothing is being duplicated why do you need a code generation tool in the first place?

That's just two cents from someone who has been burned way too many times by script-happy coders whipping up screens directly off of database tables. So pardon me if I cringe every time some mentions tools like CodeSmith.  ;)
Turtle Rustler
Friday, March 03, 2006
 
 
"After all, a good OO design results in very little (if any) duplicated code. If nothing is being duplicated why do you need a code generation tool in the first place?"

It doesn't qualify as "duplicated code", but typically form and CRUD code is highly repetitive and involves a lot of drudge work.  Integrating with a (non-passive) code generation tool is a way of removing the repetition, with rewards similar to removing duplicated code.

"That's just two cents from someone who has been burned way too many times by script-happy coders whipping up screens directly off of database tables. So pardon me if I cringe every time some mentions tools like CodeSmith."

How familiar are you with Codesmith?  It, in particular, has been designed to provide answers for every complaint you make. 

Of course, if you refuse to fully integrate the tool into your development procedures then you're not going to be able to make productive use of it or, worse, it could be counterproductive.  But to say that something is unhelpful because people are unwilling to fully buy into it is not a real strong argument. . . .
Herbert Sitz Send private email
Friday, March 03, 2006
 
 
Herbert. No, it's not a real strong argument. But it is my argument none the less. Like anything, code generation is only as good as the developer using it. All of the developers that I've personally seen using it have done so on the side and not as part of a "sanctioned" or integrated part of the development process. This is the main reason why we always end up losing the scripts.

But the idea of mundane CRUD apps violates the principle of favoring "user-centric" design over "data-centric" design. I have been very guilty of this myself in the past (and still am on occasion). It is hard for programmers to keep themselves from falling back into the data-centric design approach.

Here is a real world example to demostrate what I am talking about. From this example you should be able to see how code generation simply can't help here.

The user walks up to the computer and thinks the following: "I need to transfer all of Bob's customers to Jim. Bob and Jim are salespeople and Bob just left the company." The customer is looking for a simple screen with a title related to transfering customers between salespeople. This screen will not be a direct view into either the Customer or Salesperson tables. It is not a standard CRUD screen so a code generation tool will be of little use. But you'd be amazed at how common this scenrio is.

Note that the customer is NOT thinking: "I need to first go into salesperson maintenance and look up Bob's and Jim's salesperson ID numbers. Then I need to go into customer maintenance and query using Bob's ID value in the salesperson ID criteria field. Then I need to tab through all of the rows and replace instances of Bob's ID with Jim's ID. Then I need to click. By the way, the customers didn't change, the salespeople did. Why am I using customer maintenance again?". But this is exactly what the user ends up having to do because as developers we have simply provided grid-like CRUD screens on the table level. We designed around the data and not the user or business requirements.

So I really have two beefs with this whole topic.

1) There is rarely such a thing as a simple CRUD screen. Simple CRUD screens end up being replaced by more functional screens in the future. So why generate them in the first place?

2) Code generation should not be used to replace design. Code generation is best used for mundane and error prone tasks that need to be done often on databases and other entities that are changing often.

Again, that's just my two cents. And no, I don't have any experience with CodeSmith. I've heard plenty of good things about it so as code generators go it is probably pretty good (if you are into that sort of thing). :)
Turtle Rustler
Friday, March 03, 2006
 
 
"There is rarely such a thing as a simple CRUD screen. Simple CRUD screens end up being replaced by more functional screens in the future. So why generate them in the first place?"

With a generator like Codesmith this makes no difference.  The CRUD generator is not a black box that spits out code.  It is in fact a template that you design and which you can modify.  You can make it as simple or as complex as you want.  Also, Codesmith is an active -- not a passive -- code generator.  That means that you aren't stuck with hand-modifying 8 or 15 or 50 instances of created code if you've already generated it and want to make changes.  You go back to a single spot, the Codesmith template, make the change you want, and automatically regenerate it to the 8 or 15 ore 50 instances that are tied to that template.

It is a big switch to incorporate a tool like Codesmith fully into the development workflow.  I haven't done it myself, but I can see that it has big benefits.  Your Codesmith templates become as much a part of your "design' as the code in your primary language.  In fact, the Codesmith templates are in fact more integral parts of your design than the code instances they spit out.  It's like a meta-level of OO programming, where the source code itself is what you're working on.  Many generated code instances are generated from a single template.  The template is the critical part of your project, not the instances of code that it spits out.  The crucial thing is that generation of code is not a one-time thing that then leaves you to manually modify each instance.  If you need to make a change you modify the template and regenerate all instances.  This makes the code generation DRY in the sense meant by, e.g, Dave Thomas of The Pragmatic Programmer.  There may be times when you need to make lots of particular modifications to a particular instance.  Codesmith allows you to make these changes within the template or if it really is individual you can manually incorporate code into the generated instance that will remain unchanged when the code for that instance is re-generated.

I understand the argument that resistance to using code-generators may prevent them from being good choices.  The real question, though, should be whether that resistance is a good thing.  If it's not, then why not at least start trying to improve by working to eliminate the resistance to the code generation tool?
Herbert Sitz Send private email
Friday, March 03, 2006
 
 
I'm a fan of CodeSmith, a fairly recent convert after reading Kathleen Dollard's book, and I have to say that, like China, everything you've heard about code generation is true.  She covers the major pitfalls and ways to avoid them.  CodeSmith works well with her philosophy, but I think knowing the philosophy is better than having the right tool.

You can produce bad code with code generation.  You can produce code that can't be reproduced.  Alternately, if you write your own template, automate the code generation just like every other part of your build, keep your templates in source control just like every other part of your build, etc., then it can be a good thing.

On the other hand, I would never use it to generate a user interface.  Encapsulating a database, sure.  Exporting from a library that has metadata into an unfamiliar language that calls it through COM, sure.  But the data that goes into UI decisions just ain't there in your database metadata.  That's got to be done the old fashioned way.
Chaz Haws Send private email
Friday, March 03, 2006
 
 
Herbert and Chaz, you are both very gracious in your argument style. It is a nice change of pace from what we have been seeing our here recently. You make very compelling arguments for the use of these types of tools.

It does indeed sound like CodeSmith is much different from what I am used to seeing people use. And I must admit that I have never worked with anyone who incorporated code generation into the ongoing development process. It has always been just a temporary means to a very bad end.

I've had some very bad experiences with projects that relied too heavily on code generation. I could go on for days about the pitfalls of code generated and data-centric designs. It is one of my many pet peeves. But for the sake of the discussion I'll gladly let it go for now.
Turtle Rustler
Friday, March 03, 2006
 
 
Thanks, Turtle Rustler.  I try to be polite.  I know what you mean, though. :)

Yeah, one-shot code generation from a wizard as a starting point, then modified by hand, that sounds like how to do everything wrong.  I still shudder when I see MFC AppWizard Doc/View code that survives to this day in a production application for which there is just no useful "document" concept.

Dreamer, you should start by knowing that Turtle Rustler is dead right if you don't learn how to protect yourself.

Here's my version of the rules:
1) Everything you need to generate your code is in source control, including the code that generates code.
2) Generated code is never, ever modified by hand.  Separate the interesting, custom behaviors from the fixed, generated behaviors.

An easy way to verify these is to not even store your generated code in source control.

I strongly recommend the Kathleen Dollard book "Code Generation in Microsoft .NET" if you want to learn some details about how to implement this.  If you don't speak .NET, then you'll have some translating to do, but I think it should be manageable and I'm not aware of any other thorough treatment.
Chaz Haws Send private email
Friday, March 03, 2006
 
 
Thoughts:

- You probably shouldn't be generating GUIs using code generation.  At least, not the final GUIs.  All the problems Turtle describes are there.

- DB access you should automate to the extent possible.  Anything that must be hand coded (for example, any sort of very complicated join) should not be mixed with your autogenerated code.  It might be worthwhile marking your generated code so that you know it is generated.  I do this by putting a note at the top of the generated file.  I also put the date/time the code was generated and with what version of the generator (I have my own home grown codegen).

- Anything that you do over and over should be looked at as a candidate for code generation.  In .NET, some things that occur to me are the defining of properties and the machinary for custom events.

- Always change the template, not the generated code. 

- Practice both active and passive code generation.  If you need something huge and distinctly patterned generated, but only once, go ahead with passive.  If you have something you imagine is going to change a lot, go with active.

- Partial classes seem like they would work well for the generation of some automated code w/ some custom written code, even if they had to be in the same class.  [Note:  In the past, I have used an inheritence scheme to get around this problem.]

- Feel free to autogenerate OOP patterns.  If you have a large set of complex objects that fit into their respective OOP patterns, you might want to look into some autogeneration for their structure.  Also, if you autogenerate them you can make sure that the resulting classes are laid out in some sort of logical structure.  If that structure has to change, you can make the changes to the template and move classes from one place to another.

- Code generation is sort of like macros for Lisp in reverse.  There, you make the code, and then let the macro work on it.  In code generation you make the marco and let it spit out code that meets those requirements.

- Don't forget that you can create your own language when you are writing your code generation engine.  This will allow you to work with larger and larger pieces of information as you do your code generation.  Also, you can customize the language you write to your specific problem space (or even mimic another language that already exists in that problem space). 

Can you tell code generation is near and dear to my heart?
Joshua Volz Send private email
Saturday, March 04, 2006
 
 
"You probably shouldn't be generating GUIs using code generation.  At least, not the final GUIs.  All the problems Turtle describes are there."

I understand the point about GUI generation being different from generating CRUM code.  But I don't entirely buy into the idea that you should never use it for the final GUI.

As an example I would point to the Django web development framework.  I'm currently testing Django and Ruby on Rails and trying to decide which I want to go with.  RoR has code generation for a set of very basic forms that it calls "scaffolding".  Basically this generated set of forms makes up a temporary ui for your application that gets it going, but which you will want to modify (probably heavily) to make a production gui. 

Django, on the other hand, has a type of form generation for its administrative gui that is pretty amazing.  It automatically generates a very capable, flexible, and attractive gui for the admin section of the website, solely from setting changes you make in the Django "model".  The generated gui is so good and the settings so flexible that I'm trying to figure out how I can make it the basis of the user-facing app rather than just the admin portion of things.  It's not quite the sort of code generation that we've been talking about, but impressive nonetheless.  You can get a tiny taste of it here:

http://www.djangoproject.com/documentation/tutorial2/
Herbert Sitz Send private email
Saturday, March 04, 2006
 
 
Herbert, good points.  I think I was more talking about windows forms GUI generation when I made the statement.  I have not tired either RoR on django (btw, Is it just me or does that remind you of Jango Fett?). 

I think there are some things that you can write a code generator for and use it across projects.  An example would be the data access layer (Hibernate, etc.).  There are some things where you have to write a custom generator (or set of templates depending on your codegen) specific for the rules of that project.  I can imagine a situation where a GUI would fall into the second class.  It might be reasonable to write a GUI codegen for a specific project, but it would be harder to write one that produced final GUI objects for all projects. 

All of our dicussion on this thread notwithstanding, it seems pretty clear to me that I am more productive when I can work with larger pieces of code.  Code generation is one way to do that.  I also like it because it makes things that happen often ("Can you add this field to this screen?") very easy to implement.  I also like how it is extends the DRY principle by producing what you need from a single information source (in most cases a database).
Joshua Volz Send private email
Saturday, March 04, 2006
 
 
Joshua -- Django is actually named for the old-time jazz guitarist Django Reinhardt. 
http://en.wikipedia.org/wiki/Django_Reinhardt

Yes, I agree that nature of web forms and fact that browser actually does a lot of autopositioning does make them more appropriate for code generation than desktop forms.

For my Win32 apps I program in Delphi and use the Developer Express ExpressLayoutControl that is in many ways a big improvement for windows form design.  I use this component with sort of a hybrid code generation setup:  I use a data dictionary to create the underlying edit objects from code, and then use direct manipulation in the program gui to move each item into position and then persist the layout in a db.  This is a huge improvement over the usual tedious placement of controls on windows forms, and as a bonus allows you to let users themselves modify the forms, if you want.

The component requires a little work to get your head around, but it's worth it.  The single most common comment I hear from other developers who use it is, "I'd never go back to desiging forms the old way."  This particular component is useful to you only if you use Delphi ( http://www.devexpress.com/Products/VCL/ExLayoutControl/ ), but there is also a .NET version that is scheduled to be released soon that I think will have even more features.
Herbert Sitz Send private email
Saturday, March 04, 2006
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz