The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Why There Is No Rational Software Design Process

http://jeffsutherland.com/scrum/2005/11/why-there-is-no-rational-software.html

Jeff has some good links, including one from my name sake, David Parnas, and talks about:

"WHY WILL A SOFTWARE DESIGN PROCESS ALWAYS BE AN IDEALISATION?

We will never see a software project that proceeds in the rational way. Some of the reasons are listed below:

(1) In most cases the people who commission the building of a software system do not know exactly what they want and are unable to tell us all that they know.
(2) Even if we knew the requirements, there are many other facts that we need to know to design the software. Many of the details only become known to us as we progress in the implementation. Some of the things that we learn invalidate our design and we must backtrack. Because we try to minimize lost work, the resulting design may be one that would not result from a rational design process."

....

For all of these reasons, the picture of the software designer deriving his design in a rational, errorfree, way from a statement of requirements is quite unrealistic. No system has ever been developed in that way, and probably none ever will.
"

I agree 100%
son of parnas
Saturday, November 26, 2005
 
 
Not sure how the reasons listed differentiate software design from other complex processes like, e.g., the design and building of a skyscraper, or even of a house.  What are the differences?
Herbert Sitz Send private email
Saturday, November 26, 2005
 
 
A chip off the old block, eh?

Parnas has it from the first sentence ... "A perfectly rational person is one who always has a good reason for what he does."

Being able to know the difference between good and bad reasons is a nondeterministic process. Hence requiring humans, with all their faults.
YaYaYa Send private email
Saturday, November 26, 2005
 
 
> "Parnas has it from the first sentence .."

I wish it were me, but it was Jeff.
son of parnas
Saturday, November 26, 2005
 
 
son of parnas, YaYaYa was quoting from the Fake-It.pdf given in the link.
PNII Send private email
Saturday, November 26, 2005
 
 
> son of parnas, YaYaYa was quoting from the Fake-It.pdf given in the link.

Cool, isn't it?
son of parnas
Saturday, November 26, 2005
 
 
"deriving his design in a rational, errorfree, way from a statement of requirements is quite unrealistic"

that's only because you aren't elite enough. Now, those boys over in India, they can do all that, no problem. they are CMM level 5 certified.

They'll do it for $5000/yr salary and they'll thank you for their job. They don't need no benefits either.

Bush sucks! So do all you dumb lazy Americans!
Hyderabad Herbert
Saturday, November 26, 2005
 
 
> So do all you dumb lazy Americans!

Thankfully there aren't many of those so you invective lacks a little pop.
son of parnas
Saturday, November 26, 2005
 
 
> dumb lazy American
didn't he die a couple of months a go ???
PNII Send private email
Saturday, November 26, 2005
 
 
No, we elected him president.
Mr. Analogy {Shrinkwrap µISV} Send private email
Saturday, November 26, 2005
 
 
doh! nice catch :)
PNII Send private email
Sunday, November 27, 2005
 
 
This was an excellent read.  Thanks.
hoser Send private email
Sunday, November 27, 2005
 
 
IMNSHO using 'rational' as a synonym for 'Rational' as in RUP is invalid. I'm no fan of RUP, but I've always sen it as a superset of process, method and deliverable chunks that can be configured as appropriate for a particular project, taking account of previous projects, skills etc. Why is it a surprise to anybody that the wholesale application of RUP doesn't work? In the context of the linked article it's merely a straw man.

Ditto waterfall: while generally more restrictive than 'wild' RUP, DoD standards (for example) themselves recommended customisation, and you would have had to show how you would do it to win a Defence contract.

ISTM that the waterfall PITA projects I've been involved in have actually been stuffed by the _combination_ of poor requirements, creep, and rigid fixed-price adherence (i.e. shenanigans far away from the software/systems domain where either waterfall _or_ scrum might apply).

Anyhoo, the argument that 'waterfall is implemented poorly therefore scrum is good' is false. It's the same false argument as the creationists (sorry, 'advocates of Intelligent Design'), when they point out that scientific theories are 'only theories' and can be falsified by new evidence, therefore that answer is ID. In fact ID should provide at least as good a set of evidence as science before it can be given any serious consideration.

Likewise scrum and other agile methodologies need to justify themselves by measurable success over full-lifecycle projects (i.e. _not_ just the software component of a project built on commodity hardware).

Whatever. In the end it will be as it always has been - a handful of committed top-notch engineers who pull projects to success irrespective of process, management, latest buzzwords or the authors and consultants who make their living from pushing them.

Rich
poorHouse
Monday, November 28, 2005
 
 
> in the end it will be as it always has been -
> a handful of committed top-notch engineers who
> pull projects to success irrespective of
> process, management,

With a proper process you don't need to pull so hard, which is good because very few projects actually have a handful of comitted top-notch engineers.
son of parnas
Monday, November 28, 2005
 
 
>"Likewise scrum and other agile methodologies need to justify themselves by measurable success over full-lifecycle projects (i.e. _not_ just the software component of a project built on commodity hardware)."

Why?

Why can't a process simply be good at a small part of a project's lifecycle? If part of the problem is a bad impedance mismatch between (heavyweight) process and (large) project, then perhaps a solution is to match multiple lightweight processes to appropriate aspects of a large project.
lw Send private email
Monday, November 28, 2005
 
 
Don't be daft, man! Saying that the process has to match the task is like saying the strategy has to match the game. Oh, wait...

Monday, November 28, 2005
 
 
> With a proper process you don't need to pull so hard,
> which is good because very few projects actually have a
> handful of comitted top-notch engineers.

Are you saying that agile methodologies now allow you to throw any old programmer at a project on the basis of a half-hour daily meeting and get results?

Rich
poorHouse
Tuesday, November 29, 2005
 
 
> Why can't a process simply be good at a small part of a
> project's lifecycle? If part of the problem is a bad
> impedance mismatch between (heavyweight) process and
> (large) project, then perhaps a solution is to match
> multiple lightweight processes to appropriate aspects of
> a large project.

It _can_ be good at a small part. However the scrum bluster referred by the OP rubbishes waterfall, which is used in large projects, and suggests that scrum is the answer, when scrum is only (allegedly, at that) proven in small software-only projects. i.e. in it's own terms it seeks to replace waterfall.

There is a recent post on that site that claims scrum is now ready for large projects, citing Microsoft's interest. However the drivers for Microsoft are explicitly time-to-market, and implicitly developer hearts&minds - no mention of quality, or what kinds of project they might use it on. If it's anything like the recent 'Live' site then IMO scrum would be proved to be _not_ ready for the big time, in spite of much respect for the pople who pulled that together.

'multiple lightweight processes' sounds like a good idea. However in the agile world there are AFAIK no lightweight processes outside of the software implementation phase. People have had a go at agile documentation, but it still seems more academic than practical.

IMO however a 'heavyweight' process often does make use of a variety of lightweight _methods_ (not methodologies) within a given phase. Requirements engineering, technical studies to de-risk, deployment models, UML design, prototypes, testbeds, TDD etc. But you are still faced with an impedance mismatch between phases, where you need to define phase start/end/review criteria to be able to progress. Big process like RUP at least tries to provide a framework to enable such things... but we've already thrown RUP out, haven't we.

No, I still maintain that hard projects are pulled together by good engineers who use applicable methods and the valuable parts of a process to help, while minimising the impact of the poor parts. Real-world constraints of time and budget, and the impossibility of fully-described requirements will always apply. If a project was a success _without_ big brains then it probably wasn't very difficult, and the imposition of process of whatever flavour was merely for management information or 'house style', rather thn a necessity for achieving the technical goals.
poorHouse
Tuesday, November 29, 2005
 
 
I think it was a good article, but I don't think it ever answered the question.  The seven points mentioned simply articulate the difficulties of developing software.  (I think some of the points can be expanded to include a few things poorHouse also mentioned.)

I think software can be developed rationally - emphasis on the lower case r - but the key aspect really is flexibility.  About the only thing that I think is required, is that "what we do and why we do" needs to be written down, and at a level of detail most of us are either uncomfortable doing, or would otherwise take for granted.

Most software development and management methodologies attempt to establish some loose guidelines as to what should be written down with the assumption that if you accomplish that bare minimum, you have a substantially better chance of success than a purely chaotic approach (or one dependent upon the skills, talent and effort of an individual).

For example, the waterfall method encourages you to do all of the planning and designing work up front so that you can catch mistakes while its cheap to do so.  That doesn't mean you can't go back and revise the paperwork if you need to.

I think the problem is that many programmers try to manage their first few projects according to a book.  The results tend to be fustrating for everyone involved because they don't have the experience to bend the rules when its necessary to do so.

But back to the original subject, having some sort of plan for managing your work is a very rational thing to do.  You do want to have some idea of what you're going to code before you start, you do want to know what your constraints are, you do want to know what your schedule is, and so on.  However, accepting all these as literal gospel is the irrational part.
Anonymous Coward
Tuesday, November 29, 2005
 
 
> For example, the waterfall method encourages you to do all of the planning and designing work up front so that you can catch mistakes while its cheap to do so. 

DO WHAT!!! Rubbish. I hope you were being sarcastic. Doing all the planning and design up front means that you build in solutions based on requirements assumptions where they cannot be fully known up front (we are dealing with humans after all), and based on technical assumptions where they too cannot be fully known up front (as another contributor said, somewhere in this conversation).

The whole down side of the waterfall method is that mistakes are designed in from the beginning and are not identified until far too late, making it the most expensive time to try to fix them.
robinson Send private email
Monday, December 05, 2005
 
 
>> Doing all the planning and design up front means that you build in solutions based on requirements assumptions where they cannot be fully known up front (we are dealing with humans after all), and based on technical assumptions where they too cannot be fully known up front.

That's why waterfalls have phase-end reviews with clearly defined products and checklists, and they allow for coherent risk management at each stage. You're still implicitly asserting that up-front means 'set-in-stone', and that being in a design phase means you can't evaluate technical issues early. That isn't the case where waterfall is applied pragmatically.

The alternative seems to be an agile approach that _never_ knows about the big picture - there is no solution other than today's stories, and technical assumptions are pushed to code  and buried (sorry, 'refactored later') without visibility. The rework costs are a) hidden and b) virtually guaranteed using agile.

You're also forgetting to address the problem that agile doesn't cover system aspects, platforms, evaluating options etc. It's _only_ about writing and testing code. If I'm coding for today's story then the only test of whether I'm making good decisions is whether my code epasses the tests for the story. There's no room for me to evaluate options in the light of the bigger picture.

Rich
poorHouse
Wednesday, December 07, 2005
 
 
Just like missile defense, waterfall keeps going after failure after failure after failure. It has never worked, and never will. By work, I mean, create something solves real people's problems. The usual goal, creating something that works according to a specification by a date, is a milestone, or at least it should be.

As a culture, we just never seem to learn how to just give up and admit failure.

Good software is never done. If it is alive, it is constantly changing. Unfortunately, that makes it hard to schedule and hard to budget, which are also unfortunately necessary. Bummer.
Steve Hirsch Send private email
Wednesday, December 07, 2005
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz