The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

Is Software Engineering empiric?

Is Software Engineering empiric? If it is not, what is it then?

Is there a way to prove that a software is correct or one can only try to ensure that is correct, but cannot GUARANTEE that  it really is.

A process, like Rational Unified Process, is what? Is it an empiric process?

From Merrian-Webster, "empiric":
 Function: noun
 Etymology: Latin empiricus, from Greek empeirikos doctor  relying on experience alone, from empeiria experience, from  em- 2en- + peiran to attempt -- more at FEAR
 2 : one who relies on practical experience

Wednesday, March 30, 2005
Yes :)

All software has bugs.  There is no known way to ensure that it does not.

There are only methods to minimize bugs by doing things "right" up front, and testing the heck out of things later.

The good news is that if it were possible to guarantee that a piece of software worked correctly with no bugs, we'd all have to live up to that standard, right?

Just think: "Do you have a BugFree(TM) Certificate for this new ecommerce software?"

That's why software remains as much art as science, if not more so.
Dave C Send private email
Wednesday, March 30, 2005
Proof of correctness is hard.  Read about it here:
Chris in Edmonton Send private email
Wednesday, March 30, 2005
Some algorithms can be proven correct, mathematically. Others cannot. So yes.

Wednesday, March 30, 2005
Formal proofs of software correctness are usually an order of magnitude larger than the program. It is one of those "high brow" things that they do in "ivory towers" and folks don't, won't or can't do in "the real world." Software correctness is one of those things where "the real world" lags several decades behind academic research. Maybe 20 years from now, you won't use UML 9.6, you'll use some formal language for specifying stuff, and if the spec checker passes it, you'll compile.

Your premise is incorrect. Software engineering is not engineering. It is a craft. And calling what we do "computer science" is like calling surgery "knife science." The phrase software engineering was coined in the 1960s because people back then were tired of over-time, over-budget and too many bugs in existing software.

And the other part of your premise, that it is empirical, is also flawed. We have more fashions in software development than can be found at any mall full of teens. Example: death marches are a routine, not an exception.
Wednesday, March 30, 2005
There is a legal definition to being called an "Engineer".  There is an actual process and certification that goes along with it.  There is an actual course of study required.

I graduated with a BS in Electrical Engineering and I don't call myself an engineer for precisely that region.

Software development is NOT engineering.  Some developers and processes may be systematic about it, but this does not constitute engineering.
KC Send private email
Wednesday, March 30, 2005
If by "Empiric" you mean mostly rule-of-thumb and do what worked before, then I don't know.

We are trying to evolve Software Engineering into a true engineering discipline -- some of us, anyway.  There's an ongoing tug-of-war between the Computer Scientists who think it's a branch of Mathematics, and those people who want it to become an engineering discipline.

From my point of view, it is like we are in the early days of Civil Engineering.  We are still working out what the major models are -- in Civil they are stress and strain, and the strengths of materials.  Early metal bridges were built by rule-of-thumb, empirically.

Currently, with all the languages, and all the operating systems, and all the IDE's, and all the graphics modeling languages (UML being ascendent currently, but flawed), and all the big-M methodologies, and the fact that many Software projects suffer from over-budget, over-schedule, and/or under-functionality  -- clearly we don't have a good enough consensus on what works for it to be an engineering discipline yet.

So yes, the current state of the art is more empirical than I like.  And no, I don't think that is an inherent property of the evolving Software Engineering discipline.
Wednesday, March 30, 2005
And there is a legal implication in being certified a "Professional Engineer", or PE.  The mere term "Engineer" does not have that legal implication.  Otherwise Train Engineers would have to be PE's.  Which is ridiculous, of course.

I don't know if RF engineers, communications engineers, people who design VCR's, Televisions, DVD players, or new computers all have to be certified -- I suspect not.
Wednesday, March 30, 2005
"The mere term "Engineer" does not have that legal implication."

Depends on jurisdiciton. In my US state, you can't use engineer or engineering in your company name, unless there is a registered PE on staff, no matter what sort of engineering.
Wednesday, March 30, 2005
>The mere term "Engineer" does not have that legal implication.

Depends on the state. In some states folks with a CNE or MCSE are prohibited by law from calling themselves engineers.
Wednesday, March 30, 2005
In the UK, it is possible for software engineers to become chartered engineers. This has some form of legal status, though what that means I'm not sure.

Same is true of the rest of Europe, I believe.
Colm O'Connor Send private email
Thursday, March 31, 2005
>Formal proofs of software correctness are usually
>an order of magnitude larger than the program. It is
>one of those "high brow" things that they do in
>"ivory towers" and folks don't, won't or can't do
>in "the real world."

I *LOATHED* that course at university. We were taught a language called Z, which, it was alleged, could be used to prove that a program was 'formally' correct and thus bug-free.

Problem was, designing programs using Z was SO complex that you were BOUND to introduce a bug in the design EVEN for very, very simple programs. So the verification procedure couldn't guarantee the program's correctness anyway.

Totally pointless exercise. Partly the reason why I dislike academia. The real world agrees with me so much more.

After seeing stuff like that at university, it's not surprising that the industry was created by hobbyists, not 'computer scientists'.
Colm O'Connor Send private email
Thursday, March 31, 2005
Software correctness is not something we can prove (for complex problems). Programs are not bug free unless they are trivial.

For a more details:
Thursday, March 31, 2005
I suspect you were oversold formal methods if told that it would prove lack of bugs; you can use it to prove certain assertions hold, but not much more. Check the full set of assertions that describe the correct behaviour of your program, and lo -- you have correctness. (The circularity is intentional; that's how these things work.) Correctness doesn't necessarily correspond to defect-free, but you have proven the absence of any errors whose presence would be indicated by any of your assertions' not holding.

I vaguely remember from the formal methods course I received that our proofs ensured the dynamic behaviour of the system correct based on static analysis by limiting the potential inputs via the VDM modelling language type system. You could probably expand this by proving correct a function to convert non-deterministic elements (such as user input) into something useful for the proof, and take the proof from there.

I also vaguely remember proving things taking bloody ages :) It all seemed pretty mechanical, though, so I suspect there's scope for a great deal of automation. But not many languages are amenable to this; the modelling language we used was, but then if you're writing the final product in another language, as is likely, then you're effectively writing 2 programs. And correctness of the one does not necessarily correspond to correctness of the other.

I suspect a useful and still formally-provable language would be possible, though my intuition says that you'd dilute the ability to produce good proofs automatically to such an extent that you'd be no less worse off than we are now. (And you'd have to be able to do it automatically, because it takes ages to do it by hand.)

To top it all, proofs do require the same level of formality all the way down, and for more about that:

representative quote: "the "test a lot" model of software correctness is the truly scientific one; it doesn't make any assumptions about the relationship between the algorithm and the real-world behavior"
Thursday, March 31, 2005
One sensible-sounding criterion for true software engineering is having programs you may conveniently manipulate like any other data.

There are a few tools which do this; the most well-known being Common Lisp, and I've heard there are a couple others. I think the reality-distortion fields around our tools are decreasing to a manageable level, so perhaps these tools will help advance the state of engineering in time. Starting this year, in fact; CL has what I hear is overall the most powerful object system, because people had the ability to extend the language naturally, and hopefully there will be intriguing presentations at the OOPSLA and ECOOP conferences.

So I think a second criterion would be that programmers users may conveniently extend their tools. These are after all the most informed people, not the language designers. Democratic principles over dictatorial ones. Engineers have every possibility of creating their own theories and machines, even if they shoot themselves in the foot. Successful ideas spread; and failures are dissected if everyone's honest and alert.
Tayssir John Gabbour Send private email
Thursday, March 31, 2005
"programmers users may conveniently..." -> "programmers may conveniently..."

I'm tired. ;) Extra word there, which likely changes the meaning badly.
Tayssir John Gabbour Send private email
Thursday, March 31, 2005
"The phrase software engineering was coined in the 1960s because people back then were tired of over-time, over-budget and too many bugs in existing software."

Hmm... and software project nowdays are on time, on budget and are bug free?

Actually the field of "software engineering" has had progress, but for some strange reason the progress rarely translates out to the "real world". Personally I still cringe when ever I hear. "Would you be on time if you more people on your project?". Even before I was born it was known that adding people to a project that is late delays the project further. I constantly find myself repeating the mechanism why it delays the project. It takes time to get them up to speed. there is going to be more people talking to each other and that communication takes time away from the project.

Oh the joy...
Friday, April 01, 2005
A clarification of my view on my last post.

I may not be on target on the topic with my last post, may it be right or wrong... IMHO the "software engineering" discipline includes managment of software projects.
Friday, April 01, 2005

Your comments on Z (Zed) and formal methods was right on target. Thankfully we used Spin for our class, but it was a real bitch to get a proper boolean abstraction, even for a simple program.

It's no wonder why it's not really used as often as the so-called "computer scientists" say it should be. Hell, you'd be introducing all kinds of bugs and missing potential problems like race conditions, deadlocks and other nastiness.

The good part was that the teacher was from a local company, so he brought in a "real world" aspect to it. Although it's very useful, even he admitted that the tools are always in a state of catchup with development practices, so that compounds the problem even further. This in turns really limits their usefulness.

I remember a former coworker at a company early in my career who said that it was a craft, not a hard engineering discipline. I definitely agree with him, based on my experiences.

It would be nice if we had some kind of "certification" for software engineers, along the lines of other engineering disciplines.
Friday, April 01, 2005
Wow, QADude, be careful what you wish for.  What if the 'Certification' test included a requirement for Formal Verification?  What then?

There is a move afoot to generate a certification for Software Engineers.  The problem is there is not much consensus regarding what should be on the test.  As pointed out earlier, we do not know enough about projects using software to have much predictability.  You certainly don't want to codify today's Software ad-hoc rules-of-thumb into engineering principles blessed by a certification organization, do you?

I can only agree "it would be nice" if I felt we actually knew what should be codified.  And my position is we are still evolving that.
Saturday, April 02, 2005
I like the definition of empiric at

  1. One who is guided by practical experience rather than precepts or theory.
  2. An unqualified or dishonest practitioner; a charlatan.

I always wonder when creating a proof aren't you guided by practical experience as well? Proofs are still programming.
son of parnas
Saturday, April 02, 2005
From what I remember formal proofs are done entirely by symbolic manipulation, and don't require any experience of anything other than the list of which manipulations are valid. It is if anything helpful to think of the manipulations as being inherently meaningless, because you will otherwise go mad over how long it's taking to prove something that is intuitively obvious.

I found the following link, which gives some illustration of how this kind of thing works:
Saturday, April 02, 2005
> 2. An unqualified or dishonest practitioner; a charlatan.

I intentionally left this part off.

Just imagine if a client read this...

Saturday, April 02, 2005
> From what I remember formal proofs are done
> entirely by symbolic manipulation

Which symbols do you use? How are they applied? Where they applied correctly? My experience with doing proofs seemed to require a great deal of experience to do correctly.
son of parnas
Saturday, April 02, 2005
Incidentally, as a counterpoint I recommend Alan Kay's talk:

This is the second part, which I was more interested in than the first.

In fact, Alan is an important dissident with modern software "engineering" and computer "science." In particular, we are far far far too tempted to escape into symbolic thought, when that overemphasis is probably what got us stuck in the first place.

I suspect it may have been correct for DARPA to slash its funding; modern tech research is too ideological. Important threads were never picked up, like considering what human characteristics are most powerful in solving problems. As opposed to losing oneself in quasi-mathematics.
Tayssir John Gabbour Send private email
Saturday, April 02, 2005
I'd say software development is definately NOT empirical at the moment. I'm not sure that it could ever be, either.

It's far too fad driven, for one. It's interesting to search for empirical evidence behind, say RUP or even XP. There isn't any! I haven't found any evidence that OO is better than procedural, either. Personal preference and taste is not empirical evidence.

As for Software so-called Engineering. Pah!!!  Calling what we do 'Software Engineering' fulfills the same goal as the bookmakers who (in UK, at least) call themselves 'Turf Accountants'.

The failure rate of software projects is hardly any different to what it was before someone tried to elevate their status by pretending to be an Engineer. (Why? Artists have much more fun and more interesting lives than engineers!).

Given that the industry is 25-or-so years older than when the SE phrase was invented, surely we'd be at least a little better at it by now, whatever the approach?

One problem with being able to gain any empirical evidence is that projects are too secretive. Even within a corporation, teams hush up their disasters. It's not rare to find managers who use seriously unethical practices to match actuals to forecast plans. The result? A 25% overrun becomes 'on-time', hence the cak-handed method that went 25% over is quoted as 'best practice' and used again. How can we as an industry ever improve when such practices are common and true empiricism is virtually non-existant?
Mantissa Send private email
Monday, April 04, 2005
People would make bug free software if anyone were willing to pay for it. It isn't uncommon for there to be just a handful of errata (bugs) for hardware designs with billions of transistors. Northrop Grumman (or maybe it was Lockheed Martin) runs a software group for NASA which has had on the order of ten bugs during the past decade. But, the cost of design verification is hardware is at least 50% of the total design cost on modern designs, and the cost of verification has been trending upwards for as long as people have been making IC, and the NASA software group pays a much larger penalty to ensure their code is bug free. Who's going to pay two to five times as much to get software that doesn't have to be patched but is years behind in terms of the feature set? Besides NASA :)
Wednesday, April 06, 2005
28/w has this spot on.  People want bug-free software, but don't want to pay the extra for it.  This isn't really any different from the reasons cheap construction is used for most houses or why cars don't last 500,000 miles.
Bill Carlson
Wednesday, April 06, 2005
"Software Correctness" is a very precise term for what is, in reality, a very fuzzy judgement. Is it correct and bug-free if it satisfies the requirements? There are two problems with that:

1. The requirements are made up by fuzzy-minded human beings, and will always be mathematically incorrect.

2. Any expression of requirements that are detailed and rigorous enough for use as a "reference" to judge code will be as about complex as the code itself -- and therefore inherently buggy. (See the last decade or so of "CASE" technology.) Proving that your code is "correct" according to buggy requirements isn't useful -- the bugs are just in a different place.

3. The problem is that we, too often, build the wrong software. We identify the market, or the problem, or the users, inaccurately and without deep insight. So no matter how perfect the code is, the application is a failure.

I'm not totally against the "Software Correctness" movement -- I think compiler messages, syntax checkers, and refactoring utilities have become much better in the last decade, in part thanks to that movement. We just have to keep the term from getting overloaded.
Vineel Send private email
Saturday, April 09, 2005
Here's a problem:

* You (a software developer) design a program and prove that it works correctly

* I (your customer) then tell you, "That program isn't doing what I wanted it to do".

Your program may be 'correct': but it's still wrong, defective, because it doesn't satisfy my requirements.

To be defect-free it would need to satisfy all my requirements ... and, believe me, I have a lot of requirements ... possibly too many to enumerate?

And some of my requirements might even seem mutually contradictory.
Christopher Wells Send private email
Saturday, April 09, 2005

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz