The Joel on Software Discussion Group (CLOSED)

In case you missed the previous post, I'm writing a book on agile development for O'Reilly with my co-author, chromatic.  One of the sections has seen some heated discussion here and we've revised it accordingly.

We've just finished our last revision--find it here:

http://www.jamesshore.com/Agile-Book/no_bugs.html

This revision was based primarily on your comments.  There are a number of changes, but the biggest ones are that we added more "proof" in the form of case studies and references to research.  We also improved the organization somewhat to show that there's more to it than TDD.

This is the last thread I'll start on this topic.  ;)  Thanks for your comments!

Previous discussion here: http://discuss.joelonsoftware.com/default.asp?joel.3.430003.22

Even with the modifications, I still find myself muttering "bah, humbug" all the way through the chapter. I guess controversy like this is probably great for your marketing and promotion!

"Don't install a bug database" sounds like an excuse to do things a messy way. It becomes a case of, our bug count is low because we make it difficult to report bugs.

James,

Within about two minutes of browsing your site, I found a typo. I think you need to improve your QA!

Here's 2 more bugs: http://validator.w3.org/check?uri=http%3A%2F%2Fwww.jamesshore.com%2FAgile-Book%2Fno_bugs.html

IMO I'd tone it down a little and provide more substinance. I'm not sure mockery/sarcasm adds anything to the message, nor does the circumstantional evidence provide a very strong case.

And this just sounds like an XP consultant's wet dream:

<i>Although "No Bugs" as I describe it doesn't require super-powers, it does require that you practice nearly all of XP. XP is not appropriate for all organizations. If your team is not currently practicing XP, be sure to read section X.Y, "Adopting XP" before getting started.</i>

1.  "No bugs" issue

I'm a little concerned with this "no bugs" focus.  What you MEAN is "No known bugs in a software product delivered to the customer".  But since all your title says is "no bugs", you tend to reinforce the idea that it's possible to generate software with no bugs.

If you take that as a given (as many managers do) then the rest of your chapter is pointless.  Why implement all these actions, if all you have to do is generate software with "no bugs" in the first place?

Now, I know (and you know) that the steps you call out in your chapter are some, and perhaps all of the necessary steps to generate FEWER bugs during development, and deliver NO known bugs to the user in the final product. It's a good process you call out.

But that "no bugs" title shoots down your own argument.  You're trying to make an argument that management should invest in these procedures, and developers should follow them.  I think you need a better title.

I mean, you even say (halfway down the article)

"Don't worry! I'm not going to wave my hands and say, "Too many bugs? No problem! Just write fewer bugs!"

But you've ALREADY IMPLIED JUST THAT in your title.  Would "No bugs delivered" be a better title?

2.  "Technical Debt"

Another Issue "Technical Debt" -- what is that?  You just haul out the term in the sentence "Even with test-driven development, your software will accumulate technical debt over time."

Okay, but what do you mean by "technical debt"?  Is it BAD to "accumulate it over time"? Apparently it is, but why?  Is it what some have called "crufty code"? Is it "awkward code"?  Is it the awkwardness that code develops over time, as different design approaches are implemented in code and assumptions change?  Is it the "brittleness" (likelyhood of more breakage than fixing) code gets over time and modifications?

You must give at least an example of what you mean by "technical debt", if the rest of your argument is going to be understood.  A one sentence explanation would be sufficient, though.

And "pay down your technical debt" is a nice sentiment, but it desperately needs a "by doing xyz..." clause.  I assume you're advocating a refactoring phase, but you haven't SAID that.

Oh, and in one sentence you say "pay DOWN your debt", but in the next sentence you say "pay OFF your debt".  Since you're not using parallel language, it becomes unclear if you're talking about the same thing.

I also dislike constructions like "to continuously pay down debt in old code" -- "continuously" says you're always doing it -- where the rest of your piece says this is something you do in your 'slack' time.

3.  What "counts" as a "bug".

"Fix it now and you'll improve both quality and productivity. Fix it before its story is done and it won't even count as a bug."

There's a logic flaw here.  You advocate fixing a bug as soon as it's detected.  Fine.  But IF you fix it, it's no longer "counted" as a bug?  What was it then?  It's not a bug in the final product, true.  But it DID exist.  It DID require effort to fix it.

This is a definition of "bug" that most people don't use.  Most people assume ANY defect at any point in the coding process is a "bug".  You seem to be redefining the term to mean "any defect that remains unrepaired".  This makes productivity metrics hard to track, as work is done to fix something that ceases to exist after the work is done.

4.  Conclusion

Overall, I find your article quite excellent.  Your approach seems quite practical and useful.  I would be sorry if the fairly minor issues above detract from people seeing how useful your approach actually is.  Especially as each of them coule be easily repaired with an extra word here or a defining sentence there.

I think your latest revision is a significant improvement on previous versions. But it still doesn't build a convincing story.

The essay puts a lot of emphasis on keeping the number of outstanding defects low. This a a laudable goal. But read the following checklist, and tell me how many of your projects have even tested for most of these bugs? Are you going to create a unit test for every item on this list!?
http://blogs.msdn.com/micahel/articles/175571.aspx

"I think your latest revision is a significant improvement on previous versions. But it still doesn't build a convincing story."

What would make it convincing?

"But read the following checklist, and tell me how many of your projects have even tested for most of these bugs? Are you going to create a unit test for every item on this list!?"

That's a great list.  I wouldn't write a /unit/ test for every item on the list, but I would want every item to work.

- Some items would have unit tests--networking, exceptions, and so forth

- Some items would have integration tests--one test could reflectively check that every control has a shortcut key, for example

- Some items would be unit tested, but also visually checked in review--correct icon on alerts, for example

- Some would be first checked with exploratory testing, then programming approach changed if necessary--tablet PC support, for example

- Performance, stability, and scalability have their own special testing needs, which we'll discuss in the "Developing" chapter

- Some are only applicable if the customers want to support it, such as localization--those would be tested if/when a story for those was scheduled.

The teams I've worked with have done all of these things, but we didn't work from that list specifically.

>> What would make it convincing? <<

* An entertaining and convincing *story*. One reason why Joel is such an entertaining and convincing writer is because he creates genuinely interesting stories that draw you in, rather than just dry polemics. It's that story-telling ability that will in the end sell your point of view better than almost anything else.

* An acknowledgment that the single best way to achieve less bugs is to have really, really good people on your team. In over 27 years of seeing software development methodologies and quality assurance fads come and go, the quality of the personnel involved has for me been the most accurate predictor of the success or failure of any project.

* Less of an emphasis on XP, and more of an emphasis on agile. You're coming across (at least to me) as a bit of an XP fanatic - sorry to be so harsh!

* An acknowledgment that XP is only suitable for certain types of project, not just for certain organizations.

* An acknowledgment that XP has some significant problems, and that certain XP practices are both controversial and not for everybody. For example, you will *never* get me to pair-program outside of certain very specific situations (mentoring, knowledge transfer, and so on). This is partly for cultural reasons, partly because nearly all of my best work is done alone, and partly because I don't program like most developers.

* An acknowledgment that TDD isn't for everybody, together with a listing of its problems. Also, an acknowledgment that TDD is primarily a design methodology, not a testing methodology.

* >> Teams using TDD report that they rarely need to use a debugger. << 
A realization that this is because these teams don't understand some of the benefits that a debugger can give you. I walk through all new code with a debugger - and at least once a day, this practice catches a design or coding bug that TDD would probably never have found.

* >> Demonstrate your software to stakeholders every week and act on their feedback. <<
In the environment in which I work, the stakeholders simply don't have time to do this. Most of them need to make money - you know, that thing that allows you and I to build our castles in the sky. A good technique here is to square-root your total delivery time. So if you have 25 weeks to deliver the project, schedule a user-visible delivery every 5 weeks.

* A better description of "technical debt", and the mechanisms underlying its influence on software quality.

* A much better acknowledgment of Eric Sink's essay on life as a code economist. There are several reasons why you might want to leave dozens of bugs unfixed. In fact, the better your QA, the more bugs you're likely to find, and therefore the more bugs you're going to leave unfixed for very good economic reasons. You need to show that you've really grokked this essay.

* >> The least effective approach (but better than nothing!) is to add a manual step to your process. <<
Checklists may well be the *most* effective approach, not the least. You can apply them to every project, you don't need to write code to support them, and you can special-case them much quicker than you can special-case code. But of course the first 10 years of every software developer's professional life is spent thinking that writing more code is the best answer to just about every problem.

* An acknowledgment of the role and usefulness of up-front design in most projects. You simply cannot change code as fast as you can change a sentence or paragraph of English. Yes, many people have been brainwashed into thinking that various flavors of BDUF mean that Lovecraftian creatures will shamble drippily out of the darkness and carry them off to places of which it is not Good To Think. But it ain't so.

* Alternatively, an explanation that you're really concentrating on the benefits of XP and TDD in code quality, and deliberately relegating other processes as not part of this particular chapter (or even book).

* A discussion about the role and usefulness of design and code reviews, or even inspections (more formal reviews). And no, pair-programming isn't a replacement for reviews.

Of course, you don't need to reflect the point of views that I've glibly espoused above in your essay. But you need to have at least a throwaway remark acknowledging each point.

As well as telling a good story, you need to persuade your readers that you've studied this area in some depth, and that you're not just coming at it from a single point of view (XP and TDD really good).

Another thought, following on from my previous point - more analysis and less cheerleading.

Hi, Mark,

You're primarily saying that you would be convinced if we "acknowledge that X is true".  It sounds like you're saying that you would find us more credible if we first showed that we shared some of your beliefs about programming.

Since we do agree with some of what you wrote, that wouldn't be hard.  This is the wrong part of the book to do that, though.  We'll keep it in mind when we write Part I.

One last thing...

"Alternatively, an explanation that you're really concentrating on the benefits of XP and TDD in code quality, and deliberately relegating other processes as not part of this particular chapter (or even book)."

We are concentrating on XP practices in this part of the book (Part II, "Practicing XP").  We explain why in the preface: http://www.jamesshore.com/Agile-Book/preface.html

Thanks for your comments.

>> You're primarily saying that you would be convinced if we "acknowledge that X is true".  It sounds like you're saying that you would find us more credible if we first showed that we shared some of your beliefs about programming. <<

I think you're missing my point - of course you don't have to agree with my viewpoint. But because you're presenting such a limited viewpoint, you do need to acknowledge the existence of other opinions and more than 5 decades of research into software development quality assurance.

Now maybe you could get away with this if you developed an entertaining and interesting story to distract the reader. People like stories, and will forgive you a lot if you tell them well. But I don't see that either in this essay.