The Joel on Software Discussion Group

I'm trying to look at learning Agile development practices and start using test-driven development.  I admit that most of my programming experience has been hacks and "quick fixes", usually there is no real development environment whatsoever so this involves making changes in production, no source control, etc.

I have the book "Test-Driven Development By Example" and I've started to read it... the problem is that I never can figure out where to start when I try to write an application.  99% of the apps I write are little more than basic CRUD stuff, so there's not much else besides a bunch of data transfer objects with a service layer that reads/writes from the database.  Apart from your regular getter/setter methods, there isn't that much to the objects, and you aren't supposed to touch the database in tests.  I guess this is where mock objects come into play, and I kind of get how they work, but it's still a bit confusing because I'm hard-coding values in them and I don't get how exactly that tests things.  If my mock "service layer" just calls a mock object in the test.. what am I really testing?  The real service layer will be a lot more complex than the mock.

This is where I always get tripped up in TDD - I never know what exactly I'm trying to test or what I should be testing.  To me, I need to know what the app has to do before I can test it; writing the test first comes as insanely difficult because at that point I don't even know what the application needs to do or how it needs to do it, so how can I write a test that makes use of the API I need.. when I don't know what that API consists of yet?

Can anyone offer some pointers or books or anything else to help me get into this mindset?  I'm tired of being a hack developer and want to increase my skills.

If all you are doing is CRUD apps driven by components that you glue together (think visual basic) you might not need TDD.

I know, I know, it's heresy, but that context is radically different than how TDD was developed.

My advice is to wait for the 1% where you have to write an actual subroutine that is more than 5 lines of code; use TDD there ...

It may be that TDD isn't right for you, but you also may need to look harder at your code.

Start by trying to make sure that you never have presentation code (formatting strings, creating HTML, etc.), business logic (conditional program flow, filtering and sorting data, etc.), and data access code (creating dynamic SQL or inserting parameters into parameterized SQL, etc.) in the same class.  In fact, you shouldn't have any two of those in the same class.

If you've really gotten to that point, then you should, for example, be able to construct your presentation layer and figure out what you need to ask for from the business logic layer.  Once you know what you want to ask for from the logic, you write some tests that ask for what you want and check whether they get it.  Is it the right data, is it in the right order, do you get the right results in corner cases such as passing a null parameter?  Once you've written each test and verified that it fails (and try to make each one cover as small a bit of functionality as possible) write the code to make the test pass.

At any given point, you should only need to mock one layer.  When you're calling the business logic in a test, you pass it a mock data access layer that will feed it the right raw data for your tests to exercise the business layer.

I don't really have a book to recommend, although I've found Bob Martin's work to be very helpful.  Just keep trying it.  And you will probably not get to 100% TDD.  That's OK -- each additional module that's done with TDD will improve your life.

The type of apps I write aren't the VB-style wizard drag-and-drop type apps, but they're almost entirely data-driven and don't really have a lot of logic apart from "when the user presses a button, lookup the customer data by the ID" or "save all these fields into the database".  The rare occasions there are some custom logic, I know how to test these.. but then I feel like I'm not testing enough because I'm not hitting everything - 100% coverage and all of that.

Right now I try to properly separate the application into tiers - e.g. data models ("core") are in one package, service layers that get/put data in the DB are in another, utility features are in another, etc.  Although I'm still learning it, I try to make use of interfaces and dependency injection to keep things clean.

Test Driven Development is a huge scam.

Just like other industries (a few months ago everyone in financial servicess said there were no problems), the software industry develops conventional wisdom that is completely wrong.


TDD just vastly increases the amount of time to program anything without any benefit. The benefit of any bugs that the unit tests find are swallowed up by the additional bugs added because of the extra complexity that TDD adds to the project, and that's before we even get into the increased manpower costs.

I don't think it's a scam, just that it seems like a lot of up-front cost (despite the fact that it's touted as being the opposite of up-front design).  Also TDD seems to require second-guessing your application (e.g. "Hmm maybe it needs to do it like this") with the whole "agile" stuff being an excuse to say "Whoops, that won't work.  Let's try this instead" instead of coming up with those cases in the first place.

"This is where I always get tripped up in TDD - I never know what exactly I'm trying to test or what I should be testing.  To me, I need to know what the app has to do before I can test it; writing the test first comes as insanely difficult because at that point I don't even know what the application needs to do or how it needs to do it, so how can I write a test that makes use of the API I need.. when I don't know what that API consists of yet?"

I think I see your confusion.  TDD requires a paradigm shift.  With TDD, you are use TDD to work out your API rather than using TDD to test your API.  TDD is part of the design process.  You may already have a good idea what your design is going to look like. 

I went to a seminar once with a couple of TDD gurus and I watched them with some amazement how they started with essentially zero lines of code and ended up with very clean working code at the end.  One of the statements I remember was something along the lines of "I learned more about design patterns and design through TDD than any other method." 

I would start with business objects and try to develop your business objects with TDD.  That's really where I get the most bang for my buck (though others may disagree).

<quote>
TDD just vastly increases the amount of time to program anything without any benefit. The benefit of any bugs that the unit tests find are swallowed up by the additional bugs added because of the extra complexity that TDD adds to the project, and that's before we even get into the increased manpower costs.
</quote>

I'm going to give you the benefit of the doubt and assume you've actually done TDD, but that paragraph makes it tough.

What "extra complexity" does TDD add?

What "increased manpower" costs?

<quote>
I would start with business objects and try to develop your business objects with TDD.  That's really where I get the most bang for my buck (though others may disagree).
</quote>

For the CRUD type app that the OP was talking about, this seems like a perfectly logical and intelligent approach.

I have to say the we write enterprise applications that are essentially glorified CRUD apps in many ways and I've never been able to see the benefit of TDD.

To me TDD works great on API's of small business functions. It is easy to write test cases for methods that return single values. But the real world isn't like that. Testing the complex interactions between the different parts of large systems is incredibly hard. Things like "how does having a tax exempt item AND an item that has had tax overriden in the same transaction impact the total taxable amount"? These types of scenarios and others that are much more complex are hard to write tests for because there are an infinite number of possible outcomes.

For this reason I tend to believe that TDD is useful for a very small number of applications. I know the true believers will likely bash me for that view. But that's just how I see it.

What has always bothered me was the fact that in order to write a test and make it pass you need to know exactly what to test that is you need to have already designed the class in your mind and just test the implementation.
So strange enough I always seam to combine design first with test driven development.
The test brings us to specs .. so how do you know what to test without specs/user stories ? You need to have specs to write tests and to seed how all the tests will implement the test.
Maybe it's just me but although I am test infected I enjoy doing design first and TDD when implementing each class.
Maybe it's because I am new to this XP thing ..

That's largely how I feel, as well.

In my current situation, all of the objects are basically dumb containers for data, they don't contain any logic at all, so there's nothing to test apart from if the value set matches the value retrieved, which is a useless test.

I'm not sure how to test the implementation tiers, because those all revolve around hitting a database.  For example, if I have a method that retrieves a list of Customers, what would I need to test?  That the data layer can process queries and spit back results?  That if I call CustomerDAO.getCustomer(12345), the object it returns is really of type Customer, and the customer's name is "Bob Smith"?  Testing something like that makes no sense to test because it's from an already-tested and proven environment and I'm not actually testing anything apart from making sure that everything gets covered.

On the other hand, I definitely see the point of testing something that adds logic; for example my company sells products on Amazon and adds 15% to the total cost to compensate for Amazon's cut - something like that would be what I could see testing, to make sure that it's applied for certain products and not for all products.  Or to make sure that a discounted price is applied to a specific group of customers.  Those things I can see testing; standard stuff that's taken care of by the language you're using, not so much.

Not-So-Agile -I posted before (forgot to sign myself).. the way I see dao's testing is to have some kind of integration tests that test each Dao in isolation (such as CustomerDaoTest) and the mock the Dao's in the service layer (or upper layers) and verify that the dao's are indeed called.
This is how I approached this issue. Remember you are not suppose to test SQL you just have to test that the call to the DAO is performed and then  in another test (an integration one) test that the DAO performs as it should.
If you are coding Java and using Spring this is trivial .. I also use JMock (actually now I am switching to EasyMock).

One of the biggest advantages that I've seen with it is the improved design that typically comes out of TDD.

It's easy to quickly write code that is tightly coupled to the database and to every other module in the system. It also leads to fragility and a maintenance nightmare. TDD helps you to build your code so that is more loosely coupled. Dependency inversion ftw.

I've written a lot of code that is inherently un-testable. Virtually all of it suffers from sizable design-flaws. By writing the code to be testable, I usually have a much improved design that is more flexible and more maintainable.

YMMV, but we've had great success with it. We've dramatically improved the code quality while adding a significant amount of test coverage.

Tightly coupled code is vilified for no reason.

Tightly coupled code is easier to debug because your compiler will catch errors, and because it's a lot easier to follow the flow of the program by just looking at the code.

Fancy-schmancy designs patterns code, with multiple layers, is a nightmare of complexity.

If your system is "fragile," it was because of bad planning and design, not because you didn't use design patterns.

"TDD just vastly increases the amount of time to program anything without any benefit. The benefit of any bugs that the unit tests find are swallowed up by the additional bugs added because of the extra complexity that TDD adds to the project, and that's before we even get into the increased manpower costs."

I don't think TDD is necessarily a scam, but I do think too many people far too often take an "all or nothing" approach to it.  IMHO, that's the wrong way to use TDD or any sort of developer side testing.

I think far too often zealots and the overly hopeful put way too much responsibility and weight on unit tests.  The idea that developers (the actual engineers or programmers doing the designing and coding) should also produce unit tests that covers even 50% of their code is ridiculous and wasteful.

In my opinion developers should write focused unit tests.  That is create unit tests only for the high value or high impact areas.  Asking developers to try to cover more code is like asking the fox to watch the hen house.

Not to mention too often misinformed managers start asking their developers to also create integration tests and stress tests.  This is just wrong.  Again you are asking for trouble if you want the fox to guard the hen house!

IMHO, and I honestly believe I am right, the vast majority of tests (unit tests, integration tests, verification tests, acceptance tests, stress tests, etc) should be written by professional QA Engineers.

It makes far more sense to pit QA Engineers against the Software Engineers rather than ask the Software Engineers to police themselves.

"Tightly coupled code is easier to debug because your compiler will catch errors, and because it's a lot easier to follow the flow of the program by just looking at the code."

Ever try maintaining that? If your software never changes then tightly coupled code is ideal. But in my world, our software is constantly adapting to change. And then that tightly coupled code doesn't look so good....

"Fancy-schmancy designs patterns code, with multiple layers, is a nightmare of complexity."

Fancy-schmancy? I don't get the hostility to design patterns in general. Yes, many developers latch onto them and they become a golden hammer. But most design patterns are borne out of experience of skilled developers. Not some ivory tower academic. They're merely solutions to problems that have been solved over and over again.

"Tightly coupled code is easier to debug because your compiler will catch errors, and because it's a lot easier to follow the flow of the program by just looking at the code."

-MAX_INT for this incredibly stupid statement. I guess you must really like torturing yourself.

This is your worst nightmare. It makes debugging any problem even more difficult, especially if it's spawning a plethora of threads.

Good code can be described as "high cohesion, low coupling". I've found design patterns to be very useful in various places, but as with anything, it's not a panacea. YMMV.

"Fancy-schmancy designs patterns code, with multiple layers, is a nightmare of complexity."

What a viewpoint.  You can go overboard on anything but what is wrong with a 3-tier approach with a data access layer, a business layer and a presentation layer?  It's great for code reuse.  I have a huge project where I reuse my business objects and my data access layer in multiple applications, including two web apps, a couple of windows apps, a few web services and even several console applications.  Doing it any other way would be a nightmare.

As for design patterns, I can't imagine not using them.  I remember a time I didn't and my code was nowhere near as good.  Design patterns are a tool to use as you need to, not something to use to say the code uses design patterns.  Really studying up on design patterns and truly getting what they can, and understanding some of the basic principles that underlie just about every design pattern has made me a much better developer.

The ideal system has modules where the code inside the modules are tightly couples but the modules are completely decoupled from all other modules. Thus, whatever do you change inside a module is guaranteed to not effect any other part of your system. Developers can work on individual modules independently from each other.

The scenario above is, of course, theoretical only, because all modules have to rely on the same global store of data. You will also need various common library functions. But it's good theoretical basis from which to build a large system.

"The scenario above is, of course, theoretical only, because all modules have to rely on the same global store of data."

I think there is something you just don't understand - at least that's how I see it.  The entire point of decoupling is so modules don't depend on other modules.  My business objects have no idea they are talking to a database, or that it's sql server, yet they pass data back and forth because they use a data access layer that is decoupled.  I am able to use that same data access layer and my business objects in multiple other applications, with vastly different presentation layers or even no presentation layers in the case of console apps and web services.  Because these modules have been tested independently and are decoupled, integration is borderline trivial.

The only way you could possibly arrive at the opinions you have is you've never correctly learned and applied the practices you seek to impugn.

"If your system is "fragile," it was because of bad planning and design, not because you didn't use design patterns. "

This statement is contradictory. Design patterns are all about avoiding "bad design" Design patterns are not some flavor of the month technology. By their very definition they are proven solutions to common software design problems. Given enough time you would probably come up with them on your own, but by knowing them ahead of time it speeds up the design. If another person is reading your code and also understands the patterns they will recognize them and thus more easily understand your code.

I agree that TDD is a SCAM.

Never needed ist myself.

However, there is an easy cure: Know what you are doing.

@Thoughtless software developer

I'll give you an example: by separating business logic from DB access logic, my team has been able to switch from time-consuming manual testing of certain functionalities, which hit the DB and was fragile because sometimes the DB data changed without our knowledge, to automated tests ran each night.

I am not fanatic about unit tests, I don't aim to get 100% coverage, but I've seen myself that having a set of tests which I can run quickly, any time I like, independent of what is now going on in the database, helped my spot bugs in my code. I would not be able to do it if I had SQL layers mingled with business layer in my code.

TDD can be an exercise in nausea.  It is most confusing and nauseating to be given a feature that hasn't even been designed yet and be told to write tests before you write code.

Here are some things that may help remove some of the nausea:
1.  Code != design.  When you design a class you are making decisions about what data is going to be there and what functionality you expose.  It is OK to do this before writing tests.  Write your class variables and empty function blocks before writing tests.  (For me, realizing this took away much of the barrier to understanding TDD).

2.  Each function should have understood outputs.  That is, each function is going to write data to its return value or other class variables or a combination of both.  Perhaps the function writes values to other classes (AKA reality rears its ugly little head).  Basically, if you expect your function to write to a value outside the function put in an assertion for that.

3.  Regarding I/O, whether it be to a database or a file or a port, it's best to just have the function return the data you expect to write to I/O.  This makes it more "testable".  You can use a separate function for the actual outputting of that data.

The most difficult thing about a full-blown acceptance and transition to TDD is understanding a function in terms of what its output should be as opposed to understanding a function in terms of what it should do.  And there's the whole religious acceptance thing where you swear eternal allegiance to AGILE and have that sacred symbol thingy burnt into the back of your shoulder (gosh I hated that)...

I feel I should add some more observations about code that is produced in a TDD environment:

- You will notice few, if any, nested loops.  This may be done to ensure either faster tests or better test coverage.  Any code in an inner loop is broken out into its own function and called in the outer loop.

This leads to:
- More, smaller functions.  Some would argue this is a "better design".  Because there are so many functions, and things are broken out so much, you will finally notice

- A distinct lack of comments.  I mean these people don't even know what a comment is - they just know that they are green and ugly.  However, the tests pass so that's all good and dandy, right?

Transitioning to an organization that espouses the TDD process is especially painful.  Sometimes you know that it's just a one-line fix but NO, you must write a test first, watch it fail, and then implement the fix.

TDD is for people who can't / won't solve the hard problems of programming (ie: read Dijkstra).

I have used TDD. It's really fun to use. Until you have to solve your hard problems. Then you have to solve them *in the real world* and rewrite most of your system / code. At this point, you have to either rewrite all your tests or throw them away. But wait... Once the hard problems are solved, you don't need TDDs anymore... Functional tests are much better at this point.

I have found TDD useful to understand / reverse-engineer 3rd party APIs. Funny thing is I've never seen someone mention that.

One of the main benefit of TDD is decoupled code with well defined interfaces. But you don't need TDD to get this. And TDD is dangerous in that it encourages you to solve the easy stuff (ie: it's a bottom-up approach). Top-down is very important too. I conceed that you can mix both. The issue I have is most people jumping on TDD don't.