A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.
In a post of Joel's on this site, he discusses event based scheduling (EBS - note he also discussed this on his FogBugz world tour). One of the features is calculating the ship date of the software. While I think the idea of the feature is nice, is it really practical to try to estimate more than a few months out into the future?
Consider the case of a new software project, such as developing word processing software. In my opinion, it's foolish to try and determine ALL of the features in such a detailed way that you could calculate an accurate ship date up front. You could, at best, probably chunk it into large pieces such as "Develop the UI" and other pieces of that size and then estimate on the order of months (which will most likely be inaccurate anyway).
I'm curious as to other people's thoughts on this topic and specifically what you do to estimate the initial length it takes to build a piece of software. For example, in the "Develop the UI" scenario above, how would you even estimate the number of months that would take? I'm guessing past experience is really the best estimator.
Best practice would be to split the project into the smallest possible unit (eg individual dialog boxes) which can be relatively accurately estimated.
You then add time for testing, debugging, unforseen problems and a safety margin.
The alternative is to pull a random number out of the orifice that estiamtes come from, round that down to the next show date / financial quarter or religous festival and assume that you can beat the engineersinto delivering.
I leave it as an excercise to the reader to determine which is most common.
I will use Excel to begin with, using a simple template that groups activities and then break them down with estimates in terms of days or hours, depending on the size of the project.
I run through all the screens, reports, processes and jot down my first gut estimates.
Without considering the total efforts required, I go through pass after pass until I feel comfortable that I have considered all the work that needs to be done.
Then I look at the totals.
Add buffers for adjustments, framework bugs, refactoring, etc... all the things that increase project time that are the fault of the designer/coder. Calculate at 20% of total time.
Add buffer for misunderstandings of requirements, minor adjustments from the client - I buffer again 20% of the total est time.
Add buffer for first draft revisions, 20% of total est time.
Add buffer for second draft revisions, 20% of total est time.
Add buffer for final draft revisions, 20% of total est time.
Add time for BETA deployment, installation, training - 20% of total est time.
Add time for BETA adjustments. 20% of total time
Example, small application with 15 tables, 5 screens, 2 reports, simple backup/restore, simple db maintenance.
Initial work effort estimate 50 hours
Framework adj/system test 10 hours
Requirements adj/revisions 10 hours
1st draft revisions 10 hours
2nd draft revisions 10 hours
Final draft revisions 10 hours
BETA deployment, training 10 hours
BETA adjustments 10 hours
Total estimate: 120 hours
When I was a newbie programmer, I would have quoted at 50 hours and would often be 'late'.
It's not a perfect formula, but it works for me.
Tuesday, December 18, 2007
I believe this is an ongoing debate. On the one hand, you have EBS, and The Personal Software Process (PSP), and Boehm's COCOMO estimation technique, all of which try to use existing code and previous projects to develop metrics and rules-of-thumb for predicting the next project.
It SHOULD be possible. After all, Software Engineering is an Engineering discipline, based on sound principles like Information Hiding, Coupling and Cohesion, OO Patterns, etc. If we are to be taken seriously as an Engineering Discipline, there MUST be something predictable about the process. A builder who couldn't tell you how long it will take to build your house (based on the blue-prints, and earlier houses he'd built) you would think an incompetent builder.
On the other hand are the Agile Process people (including XP) who point out that EVERY software product is radically different from every other software product -- different implementations for the same application domain, and wildly different implementations across different application domains.
Their response to this is to plan out explicit detail only a few months at a time -- and to leave 'general detail' outside that space.
So that's the two 'major' approaches. It's complicated by the need for Contracting organizations to specify Cost and Schedule in order to get the Contract. And for that to happen, the Customer has to be told a complete Cost and Schedule for the project that he'll believe, and be willing to pay for.
And THAT only works in software if you already have product on the shelf (built on speculation) (very rare, and very risky, and expensive if it never sells) -- OR you have a repeatable process that you can make predictions about. OR you're extremely lucky.
Thus the emphasis on SOME "repeatable process". But we're still evolving what an accurate "repeatable process" looks like, because all earlier processess still demonstrate they have limitations.
Tuesday, December 18, 2007
A common theme seems to be breaking down tasks into hours/days, which seems like a good idea. But, it is pretty much impossible to do this up front for a long term project (i.e. more than even a few months), wouldn't you agree? Especially when you're talking down to the level of individual dialog boxes.
One of the problems new managers get into is saying "because I can't estimate the completion date perfectly I'm not going to estimate it at all". The longer your project the greater the errors will be, but you should be able to put some bounds on your ship date, even for a project that will last years. As you get further into the project the bounds on the ship date will come down. Giving no kind of estimate prevents those people who depend on knowing the ship date from doing their job.
Tuesday, December 18, 2007
I agree 100%. The only point that I was making is that you can't estimate in such small time frames up front. Appreciate your thoughts!
Tuesday, December 18, 2007
>> is it really practical to try to estimate more than a few months out into the future?
No, but it doesn't matter. If you want someone to fund your development (internal or external), they have to have some idea of what it's going to cost** to make their value judgment about whether or not to go forward. Sooner or later, they're going to want to know _when_ and very few places will allow "it'll be ready when it's ready" answers forever.
>> such as developing word processing software
Almost nothing you do will be completely greenfielded - someone will have some idea of what they want and something they'll point at to say 'this' or 'not this'. There's your basis for features to plan/estimate. Depending on how unsure they are, it may take one to n iterations to get a reasonable definition. After you've got your 'reasonable' definition you can start estimating*** and _communicating_ progress or lack thereof. Jason's inclusion of estimation time _in the estimate_ is important IMO because it makes the client realize that all parts of the job cost something.
If you have no estimate, you have nothing to gauge progress against. This is how you end up months late and with doubled costs. Regardless of what you though your schedule/budget are/were, the people you're delivering to will probably have expected faster/cheaper unless you have been right up front with them - continually.
>> I'm guessing past experience is really the best estimator.
Unfortunately, unless you're building exactly the same system again, past experience is only a guide to what to expect. If you read McConnell, his book on software estimation uses a funnel as an anlogy. The earlier in the project, the wider the uncertainty - the later, the narrower. Experience does count, because it allows you reduce the uncertainties and make the funnel narrower at the start. Highly recommended if you're going to be doing this regularly.
>> A common theme seems to be breaking down tasks into hours/days, which seems like a good idea. But, it is pretty much impossible to do this up front for a long term project (i.e. more than even a few months)
Yes it is. But the estimate for a project that long is not a one time activity. It needs to adjust to reality-things won't be as assumed, some task estimates will be wrong, scope changes will happen, your key person will get hit by a bus, et cetera. The estimate has to adapt or it will become less and less relevant to the reality of delivery. The dictum "No plan survives first contact with the enemy" is true in software.
Having been through this a few times, as both a developer and a team lead, whatever estimating approach you take (nothing, Kentucky windage, or DOD-style complete), __communicate__ with your clients/stakeholders. Failing to deliver bad news early will more-than-likely mean delivering Really Bad News later.
** time and/or money, and time _is_ money
*** good luck getting the client to understand that _estimate_ does not mean _guarantee_.
Thank you for such a detailed response. I think you've cleared up quite a few of my questions. To change the topic a little bit (but still keeping on the topic of scheduling), I'd like to look at scheduling for an agile perspective.
According to most agile processes, you really only plan out 1 or 2 iterations into the future. Let's assume we have a 1 year project and detailed plans for the next 2 iterations. Is it then feasible for future iterations (in the initial schedule) to say something like "UI - 3 Months", "Spell Check Engine - 2 months", etc. And then break it down as I get closer to the actual iteration? My UI and Spell Check estimates would be based on my set of requirements, so at least I could get a ball park estimate of approximately how long the software would take.
Do what I've done for the past fifteen years (or had done to me by employers): pick a date, and ship your software on that date.
Not done? Get done!
Tuesday, December 18, 2007
You've got two ways to do this. In the Agile way you just keep doing the work, iteration by iteration. But because you haven't done any design work up front you don't know if some of those "one day" tasks are going to turn into a ten day task, or spawn ten other tasks. So you don't have a good idea of when you will be done until nearly at the end.
If you do the design up front (BDUF as they say) you will have a much better idea of what is involved and be able to estimate better. Not that your estimation is going to be perfect, but it should converge much sooner than the Agile version.
Which way you choose to go is the subject of a whole acrimonious debate across the web, but at least one of the factors should be how important it is to get your ship date estimate correct.
P.S. Whichever way you go, always make sure you have a fallback for what you are going to do if - sorry, when - you find you've underestimated the amount of work.
Thursday, December 20, 2007
Sorry, Clay, I have to disagree with your statement about BDUF converging sooner than agile.
I find the overdesign and gold plating in most BDUF designs more than offsets any refactoring or design adjustments required in most agile releases. Besides, a project that isn't doing ANY design is not really an agile project. An agile project is doing just enough design and, sometimes, just enough design includes designing in anticipation of things a few months down the line. This is probably one of the biggest misconceptions about agile, in my experience.
Anyway, back to agile ship date estimation. Agile is intended to be an empirical process. What I mean by that is you look at how much work you did last month, look at how much work you believe you have left, and extrapolate from there.
All of the agile projects I've been involved with have had a pretty good idea of the scope they wanted included in a release. It hasn't been all that hard to come up with a tentative release date.
Thursday, December 20, 2007
I understand what you're saying about see how much work you did last month and extrapolate, but the scenario I keep going back to in my mind is this. I'm the lead developer/project manager at a software shop, and a new customer comes to me to have a software project developed. I don't think it's fair to say to them, "I'll let you know sometime several months into the future how long this will take." (I don't think they'd be my customer after that). In an agile environment, you really hammer out the details only a couple of weeks in advance. So, how am I supposed to even ballpark how long it will take without looking at the details? Sounds like "which came first, the chicken or the egg?" Thanks!
OP - Are you doing shrinkwrap, consulting, or working in a corporate environment with an 'internal customer'? Makes a difference in expecations and ramifications.
You seem to be getting hung up on the "accurate" and "long term" parts. In my experience, these are mutually antagonistic, if not mutually exclusive. The further out you go, the less likey any one part of your estimate will be accurate and correct. Just like your local weather man.
You and your client(s) are going to have to come to some agreement on how much uncertainty you can each live with and work within that boundary. As your team gains experience, it's almost guaranteed their estimates will get better - if you track your assumptions, estimate vs actual and the like and let the team members know how well they're tracking.
>> I don't think it's fair to say to them, "I'll let you know sometime several months into the future how long this will take."
But, as has already pointed out, you need to have some agreement with your client on what their requirements are. The places where things are truly unknown, you'll have to do a best-effort estimate and when you communicate it, say 'Our expectations are X, Y, and Z. Given the unknowns, our estimate could be off by <-10%> to <+300%>. By <start date-[30|10|whatever]>, a final requirements analysis will need to be agreed on and the delivery of ItemX can be scheduled'. This approach is how most of the big consulting shops approach long term (> 1 year) projects.
As Bruce points out, and I think this is true of either Agile or BDUF, you need to revisit the pending-work schedule/estimates continually using your empirical data. With some effort, you'll get better over time. Expecting 'perfect' out of the gate is unrealistic.
Here's what I have done when providing an estimated ship date for an agile project:
1. Create the project backlog. Use story points, not real time, to estimate them. The time to do something will change as you learn more, but the relative difficulty of one story compared to another is more enduring, I find.
2. Unless you have some experience with your team, you're going to have to "guess" at their initial velocity.
3. I usually also produce a worst-case and best-case velocity. The range between these will cover your most likely release dates. If your new project involves a radically new approach, architecture, etc or is way outside your normal expertise, go with the worst case. If you're repeating something well known, you might want to consider best case.
4. Now it's negotiation time. Basically, agile planning boils down to this:
- I can spend x amount of time drilling down into the project, create a work breakdown schedule, create Gantt charts, move resources around, and try to come up with a more reliable date. x in this case could be weeks or even months.
- I can start a sprint/iteration now, get a walking skeleton working, and try to attack the areas where we're the least confident. At the end of that sprint you will have a start to the project and we'll have a learned a bit more that will allow us to tune our schedule and estimated ship date.
5. After start, I find the burndown chart is the most powerful tool in your arsenal. Stakeholders will not like it at all if the project backlog isn't burning down as fast as planned but they also can't ignore it in the way that sometimes happens with traditional project plans.
Friday, December 21, 2007
Sorry, one more thing. If you like, there is a bit of sleight of hand in agile planning when dealing with customers.
New customers who don't know you (and therefore don't trust you) are naturally primarily interested in the release date. I would be too. The sleight of hand in agile is just getting them to agree to invest in a month, or two, or less so that you can build that walking skeleton and provide a schedule that's based on more than just guesses.
Most customers, when they see their project starting to take shape after as little as a month, forget that you don't have a big MS Project schedule and use strange terms like velocity. I guess we're trying to get them invested as partners in a project rather than negotiators over a schedule "contract" that isn't worth the paper it's written on anyway.
Some may call that trickery. I don't know. All I know is that we, collectively, spend an awful lot of time arguing over schedules at the beginning of a project as if they actually had a chance of becoming real (note that I'm not arguing against estimating, just scheduling many months in advance). I WANT to provide a customer with a date they can depend on. I'd just rather do it by basing it on actual empirical evidence.
Of course, if you get the customer to buy into it, then you HAVE to deliver on those first few sprints/iterations, but that's the fun of it, isn't it?
Friday, December 21, 2007
"In my opinion, it's foolish to try and determine ALL of the features in such a detailed way that you could calculate an accurate ship date up front."
Jeff - If you start with that as your starting axiom, yes, you can show that scheduling is pointless and inaccurate. Everything you said after that point is not worth discussing because it relies on a fallacious and foolish assumption.
Obviously you do not know how to do accurate scheduling, but others do. Instead of improving your skills, you seek to prove that the skills of more qualified people to not exist. This is a political maneuver not a technical one and therefore belongs on slashdot rather than this forum.
Tuesday, December 25, 2007
This topic is archived. No further replies will be accepted.Other recent topics
Powered by FogBugz