The Joel on Software Discussion Group (CLOSED)

A place to discuss Joel on Software. Now closed.

This community works best when people use their real names. Please register for a free account.

Other Groups:
Joel on Software
Business of Software
Design of Software (CLOSED)
.NET Questions (CLOSED)
TechInterview.org
CityDesk
FogBugz
Fog Creek Copilot


The Old Forum


Your hosts:
Albert D. Kallal
Li-Fan Chen
Stephen Jones

over-engineering

In my opinion, creating extensive class hierarchies for an imagined future problem that does not yet exist, is a kind of over-engineering, and is therefore, bad.  I'm curious what others think.

Let's say the client has a requirement for an AnimalManager.  Currently, the application uses Dog and Cat.  Simple, right?  Well, how many times have you seen someone create an Animal interface (this one is probably needed), AbstractCanine and AbstractFeline base classes, Biped interface, etc.?  Do you see what I mean by over-engineering?

I think I fully understand the concepts behind OO, but having to discuss this problem with people who like creating these deep trees, I get the sense that they think I'm somehow ignorant of OO concepts because I don't acknowledge the need for all this abstraction.  My philosophy is to solve today's problem, then refactor as needed to solve tomorrow's problems.  Anyone else agree?  Disagree?
Simpleton
Tuesday, September 19, 2006
 
 
They think they will need this stuff because their Animal model will change in some unforseen way, so they try to create an uber-flexible model which can handle any scenario.

90% of the time the Animal model never changes, just the color of the text you use to display it.
Sassy Send private email
Tuesday, September 19, 2006
 
 
What you are experiencing is what happens when you let USERS take total control of the spec process.

I NEVER experience what you experience because I TELL them what will be developed based on the specs they provide.

I also don't let users dictate how the business logic within an app or the UI will be designed either.

Once in a while someone will make a SUGGESTION as to what they think will be appropriate within a UI but it is just that - a suggestion which I may or may not ultimately include in the app.

When you develop an app you have to be a strong DIRECTOR of how it will be with input and feedback from users. Your job as the developer is to interpret what users are asking for into a powerful and useful tool that woos them into a sense of acceptance.

The specs drive the UI in many ways. It's really not that difficult.

Once you begin being passive when it comes to specs, it's the beginning of the end...
Brice Richard Send private email
Tuesday, September 19, 2006
 
 
I think this is a very common problem esp. amongst juniors, and with respect to BR I don't think it has anything to do with users at all. I think its a developer problem. I like the term "mental masturbation" to describe this sort of thinking. XP talks a lot about this... "the simplest thing that could possibly work" etc.
Greg Send private email
Tuesday, September 19, 2006
 
 
No, 'Simpleton' is describing what happens when over-zealous "OO" designers get their mits on things.  That, or when their MANAGERS get their mits on things.

Shoot, the customer doesn't care how deep or flexible your inheritance hierarchy is.  They just want it to run.

I've seen this before, where a relatively simple 1500 line program explodes with added sub-classes and OO theorizing into a 5,000 line program -- that does basically the same thing.
AllanL5
Tuesday, September 19, 2006
 
 
Yeah, I'm not sure where the customer/user angle enters this either. It seems like more of a developer issue to me.

Someone, I believe Alan Cooper, once said that developers recognize only 3 numbers: 0, 1, and infinity. If they know the number of instances of something is not zero or one, they tend to design for infinity. In the same way, if you have multiple derived classes, developer logic tends to tell you that more will likely be needed and therefore you have to design for that likelihood.
Bruce Rennie
Tuesday, September 19, 2006
 
 
Agree with you all the way. Over engineering is a problem not only in OO design (when you look specifically into programming aspects) but also in system design (multiple components).

The most common tendency is to design for that 2% possiblility at the expense of the 98% reality. One can always argue that by ignoring the "distant-future" scenarios one would compromise design. In my experience, I've rarely, if ever, seen it happen (and I'm not talking about designing space shuttles). If the software has to live for that many years, there's a much better change that you will refactor it far better 2 years down the line with your then-improved knowledge (based on deployment experience, self growth, tool improvements and so on) than you would be able to design now.

Without doubt, it's important to design with modularity and extensibility in mind - but not to the extent that your design is run over by distant hypothetical what-ifs.
v
Tuesday, September 19, 2006
 
 
These were developers (and our manager), not end users.  We had a code review recently and that's why I brought this up.

In another example, I had to modify a 5-level deep class hierarchy with only branch.  Did it ever occur to the developer that this should be collapsed into one or two classes and an interface?  It could have been, and would have made it easier to understand the design and make the necessary changes.  Instead, he complained about my changes to the base class (which he created), affecting all the subclasses.  Duh, that's what happens when you use inheritance, kid.  If Foo is not a kind of Bar, then why did it inherit from it?

My main gripe about our profession is this kind of religion.  Each camp (Java, .NET, LAMP, etc) has its own variety of it.  There is a high degree of herding in software (like fund managers on the latest investment fad).  It's not based on objective truth, just personal preference and a need to be validated by others with the same opinion.  I appreciate elegant and abstract designs, but not at the expense of other more practical considerations.

This is another reason why I want my own ISV.
Simpleton Send private email
Tuesday, September 19, 2006
 
 
Daniel
Tuesday, September 19, 2006
 
 
Most people (myself included) get carried away with inheritance the first time they discover it.

Now, my mantra is "You Ain't Gonna Need It"
Mike S Send private email
Tuesday, September 19, 2006
 
 
"there's a much better change that you will refactor it far better 2 years down the line with your then-improved knowledge (based on deployment experience, self growth, tool improvements and so on) than you would be able to design now"

Well done "v"!

I don't think I've heard a better statement of this issue - which really is a pretty serious one! I've actually seen more projects get into trouble for *over* engineering (in this sense) than under engineering.
arethuza Send private email
Tuesday, September 19, 2006
 
 
I believe this kind of thing is common with inexperienced developers who like to try to implement all the OO theory all the time. What's worse than these are zealots and overcompensating insecure geek types. You know, guys who have to prove how much smarter they are than everyone else around them.

One guy I worked with was a design pattern freak. These are the worst in my opinion. Class hierarchies I can deal with but patterns can REALLY obfuscate a code base.

I once overheard this guy mentioning to a colleague how he regretted not having made factories for all his classes in the decently sized project we were working on. True story. Factories for ALL his classes.

He also followed the common design pattern freak custom of always specifically naming the pattern he was using (you know, so everyone knew that he knew) no matter how ill-fitting. A Decorator was a Decorator, no matter that it has nothing to do with the UI situation which coined the term. Dude, how hard is it to understand the concept of a class that holds an instance of some other class and implements some of its interface? Calling your HashMap a Decorator just doesn't help anybody.

Mental masturbation, ego-driven development, "look how smart I am", many names for the same syndrom.
jz Send private email
Tuesday, September 19, 2006
 
 
Nice question.

From my own experience, over-engineering is bad, but not because programming with greater care for the tomorrow is bad, but rather because the over-engineering in most cases does not help a lot in achieving the goals of the project, mainly because not everybody is on the same page and mainly because the deadline is "too soon" for all of those over-engineering techniques to pay off.

My experience tells me that hardly ever will any kind of over-engineering pay off in only a project, unless it's a project of live or die scope, like in a startup.

As an industry, the average is the average. No over-engineering will make it any less average. On the contrary, unless proven otherwise, over-engineering will make it average; after all, everybody tries.

And to make matters more difficult, Object Orientation is not the "end all, be all". Nowadays I consider OO important, fundamental, along with Functional Programming. Productivity in a pure OO world should suck bigtime, and no over-engineering will improve it.

Average sucks.
Lostacular
Tuesday, September 19, 2006
 
 
"Let's say the client has a requirement for an AnimalManager.  Currently, the application uses Dog and Cat.  Simple, right?  Well, how many times have you seen someone create an Animal interface (this one is probably needed), AbstractCanine and AbstractFeline base classes, Biped interface, etc.?  Do you see what I mean by over-engineering?"

Sounds like Intelligent Design.
KC Send private email
Tuesday, September 19, 2006
 
 
What do you mean KC?
lenny
Tuesday, September 19, 2006
 
 

Tuesday, September 19, 2006
 
 
Design for the future is as bad as design for scalability or design for performance. Object oriented or not. It is all the same because nobody knows exactly what to design for until the future becomes present and you know how the system changes and / or which components are bottlnecks and need to scale up or improve in performance.

Unless of course you have crystal ball which tells you what the future will be. Mine is broken. So I stick to what I know: the present reality.
Dino Send private email
Tuesday, September 19, 2006
 
 
"factories for all his classes"

I think I had to maintain code "designed" by the same guy!

Every single class had an interface, a factory interface and a factory class. This achieved nothing apart from a huge amount of obfuscation and indirection.
arethuza Send private email
Tuesday, September 19, 2006
 
 
Definitely "Yagni" -- You ain't gonna need it.  Do what is needed at the time, to solve the problem in front of you.  Okay, MAYBE look one or two steps ahead to 'slant' your solution.  But that's it. 

This "Engineering because we MIGHT need it in the future" results in excruciatingly connected code that just gets thrown out in the future anyway.
AllanL5
Tuesday, September 19, 2006
 
 
Also, realize that this industry encourages obfuscation.  If YOU can be the 'star coder' who implements really complicated solutions, then YOU can be irreplaceable.  No easily replaced 'jelly-bean coder' position for you, no sir.
AllanL5
Tuesday, September 19, 2006
 
 
Simpleton, I think you are on the right track with your skepticism.  If there really is a business need to process Animals or Bipeds generically, let the business requirements demonstrate this need.
Flow Send private email
Tuesday, September 19, 2006
 
 
Theres a difference between over-engineering and mis-engineering.

Overdoing something means making it, in all respects, way better than it needs to be. Bridges, for example, can withstand a lot more weight than their official weight rating. Cars can drive faster than the speed limit and run pretty well on less than perfect gas. Sure, this is wasteful, but it's not exactly sinful. As the quote goes, "anything worth doing is worth overdoing."

But misengineering is much worse. It's focusing on improving one aspect of a system and ignoring something else. For example: almost any software application with a great engine and terrible UI. Longfellow tells it best "In the elder days of art/builders wrought with greatest care/on each minute and unseen part/for the gods see everywhere."
Robby Slaughter Send private email
Tuesday, September 19, 2006
 
 
"If YOU can be the 'star coder' who implements really complicated solutions, then YOU can be irreplaceable.  No easily replaced 'jelly-bean coder' position for you, no sir."

Sad but true.  I'm still hoping some companies out there recognizes the difference between effective engineer and code jockey.  But as I gain more business experience I can see this is looking less and less realistic.
Simpleton Send private email
Tuesday, September 19, 2006
 
 
"creating extensive class hierarchies for an imagined future problem that does not yet exist, is a kind of over-engineering"

Probably so most of the time. YAGNI => "You aren't going to need it."

"and is therefore, bad"

Depends.
Meghraj Reddy
Wednesday, September 20, 2006
 
 
Robby's right.  This isn't overengineering, it's misengineering.

It's a common misconception that the purpose of inheritence and polymorphism is code reuse.  It absolutely is not.  You can often get better results much easier by using composition instead.  No, the purpose of polymorphism is to decouple interface from implementation so that similar objects can be manipulated genericly.  You have no business subclassing an object if you don't understand the Liskov substitution principle and the open-closed principle.  And that's what's really going on here in this example.

Now, my overengineering pet peeve is unnecessary optimization -- usually in the form of reinventing the wheel (writing one's own memory manager, creating memory object pools for no good reason, writing one's own collection classes).  Or a perverse tendency to cache easily-calculated values rather than recalculating them as needed -- this usually causes horrible temporal coupling as well as being a gross violation of the DRY principle, where the same information is being maintained multiple places in the system and they all need to keep in sync.  To paraphrase jwz, solving a performance problem with "I know, I'll add a cache" typically results in you having two problems.  This one I see all the time, much more frequently than elaborate inheritence hierarchies.
Alyosha` Send private email
Wednesday, September 20, 2006
 
 
Sure, over-engineering is bad, but the flip side of that coin is working on a code base where the simplest thing possible has been done, time and time again over several years.

The result is, for example, 2000 lines of code in one function and ifs 5 levels deep. That's why I have a problem with just saying 'do the simplest thing possible'. Some people seem to use 'KISS' as a licence to keep adding hack after hack to 'simple' code.

The best case is design with the flexibility that the problem demands, but don't overengieer as you can refactor when needed. Unfortunately, that doesn't make for a snappy ackronym like KISS.
Anon for this
Wednesday, September 20, 2006
 
 
There's nothing wrong with long inheritance trees with no branches, provided that at each stage the resultant class is a) conceptually intact and b) has a test suite.

They show up in my project where we're building a fairly big class, but I really don't want to write all the code in one go and I'd like some intermediate objects which can then be tested independently of the extra stuff.

Typically they'll still only be a few layers deep; the first few write a generic object which has re-use potential. The second set add the specificity required for the app we're writing. This doesn't violate YAGNI as long as you don't goldplate the lower classes (or their tests). If you reuse them, goldplate them then; otherwise, they're just starter classes.

People typically write huge classes when the cost of starting a new class is large[1]. If that cost is nothing, there's no more reason to write all your code in one huge class than there is to write all your code in one huge function; separating the class logic at sensible boundaries can make a lot of sense.

It can be done badly and generate dense hierarchies where the logic is too smeared to find. But decisions on development should not be taken on the basis of deprecating any technique which can be misused by bad developers.



[1] Things like "not having a tool which writes the boilerplate", "having to get approval for the class name" and "having to faff about with the makefile to get it built" are all ways of adding to that cost that people seem astonishingly keen on.
Katie Lucas
Wednesday, September 20, 2006
 
 
I hate over-engineered class hierarchies which are so deep and so complex that they need mappers in order to communicate with the database. Some developers start with their class hierarchies and then design the database as an afterthought. This is a recipe for disaster.

My approach, which some OO zealots consider so bad that they want me burnt at the stake, is to start with the database, which must be properly normalised. Then you build one class for each database table. Simple. Less confusing. No need for stupid mappers. And it works!

Using this technique I have automated the building of classes by constructing my own data dictionary. I build the database, import the structure into my dictionary, then press the "export" button which automatically creates a class file for each table, plus a separate file for each table structure. If a database table is changed for whatever reason I simply re-import into the dictionary (this synchronises, not overwrites) then export into the application, which overwrites any structure files but leaves any existing classes intact so as not to lose any customisations.

As each physical table class is an extension of an abstract table class I therefore end up with a class hierachy which is no more than two levels deep. Simple but effective. Anything else is over engineered.
Tony Marston Send private email
Wednesday, September 20, 2006
 
 
> Someone, I believe Alan Cooper, once said that developers recognize only 3 numbers: 0, 1, and infinity

lol! I misread that. I was thinking "what does Alice Cooper know about software?"

Wednesday, September 20, 2006
 
 
if you want to get management on your side you might try explaining YAGNI in terms of risk reduction. in the more general case (ooo the irony) if:

n is the cost of designing a feature now
f is the cost of doing it in the future
p is the probability that you will need the feature in the future

then you should only do it now if n < fp.

this should neatly give you a management style argument for not doing anything if f == p or if p is very small, which should cover most overengineering your coworkers want to indulge in
jk
Wednesday, September 20, 2006
 
 
Except that your formula is too complicated for most managers to understand.
Steve Prefontaine
Wednesday, September 20, 2006
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz