Why I think test driven development suck
Well I figured a provocative title like that would catch your attention, so please read on to find out I am not as stupid as you might think.
Unit tests are great, they reduce unknown bugs and arguably lead to better object design. The reasons for this are simple; Well thought out unit tests will force you to think about the input and output of any method your objects may have, forcing you to correctly decomposition the needed functionality within your object, because methods that have too much responsibility are a pain in the ass to write unit tests for. Simple one task methods however are easy to write unit tests for. Because your writing unit tests you are (hopefully) actively thinking about what can go wrong so that methods can actually fail.
So taking for granted that the above is true, why does test driven development suck?
Well the answer to that is a market one and really only applies to professional business. When a client comes to your business and asks for a offer for his project, you will quote him X, where X is roughly the amount of hours it takes to create the unit tests and then fill in the code + other stuff.
A development house that doesn’t use test driven development will quote that same client only for the time it takes to write the code + other stuff. We all know that after that he will spend a few weeks/months bugfixing, but that’s not what he quotes or tells the client.
Naturally the client will more often then not choose your competitor, unless you can positively convince him that the extra hours you spend will actually save him money in the long run. Which from the clients standpoint is a bit of an unproven claim if he is not technically inclined. So from a business standpoint, test driven development sucks as a selling tool.
Secondly, while this heavily depends on the morality of your business, those bugfix hours can be profitable. Depending on how smooth your talk is, you could very well be billing the client for those hours.
Thirdly, test driven development often suffers when deadlines need to be met. We all know that estimated hours is an imperfect science and often goes wrong, be it overconfident developers or overselling sales people, deadlines tend to be missed in our industry. So when deadlines are creeping, unit tests are often one of the first things to be dropped off schedule. Yes, we all know that shouldn’t happen, but it does. Once this has happened and no extra time is allocated to bring the tests up-to-date again (good luck getting that approved), the test driven development has failed and you’re just plainly developing again.
So for these reasons I think test driven development sucks. I might sometimes use it for private projects or open-source projects (for which I think it is absolutely brilliant), but certainly not for business ones. With the possible exception of in-house developed products, but I haven’t had much experience with that.
For anyone who now still thinks TDD is a sound development methodology, I would be very interested to hear you thoughts.
Yes, I still think TDD is a sound development methodology, and here are my thoughts:
My team uses TDD now for about 2 years consequently. Our experience is that we develop software now faster than in the times without TDD.
The main reasons are:
- almost no time wasted debugging
- less defects (also during development time which counts to the fixed price the customer pays for, therefore we benefit)
- the development pace stays high throughout the whole development time (without TDD the pace goes normally down with each month because new features cannot be added easily)
However – and in this point i have to agree with you – if you are in the transition to learn TDD, there is more cost, more needed time.
In your second to last paragraph you start to confuse TDD with unittesting in general. Because in TDD you are testing *from the start* and *before the code*, there is no possibility to drop testing when a deadline looms. This is one of the benefits of TDD over regular at-the-end testing.
I still think TDD is valid. You do forget that even though you add time for the tests to your quote, you will probably remove lots of development time from your quote. Because in non-TDD, you will have to take into account the bugfixing and a lot of “manual testing” time that is done before a project is delivered to the client and you get into the “possibly billable bugfixing” phase. Also, I don’t think many clients will actually like to receive an invoice for fixing clear bugs, so you’ll also have to take into consideration that for many projects you’ll have to spend much more time that isn’t billable.
And you also forget one very important fact: Many enterprise-level projects simply require unit tests.
Ok so basically, my claim that TDD initially takes longer is not true, or at least not true for teams that are sufficiently experienced in using TDD.
I didn’t pull the idea that TDD takes longer out of thin air though, in 2008 there was a study by MS and IBM, that asserted that implementing TDD increases the initial development time with around 15-35%.
http://channel9.msdn.com/posts/Peli/Experimental-study-about-Test-Driven-Development/
But I do think those studies where made with teams that may have been new to working with TDD, so the hit you take in time might very well well become less and less with more experienced developers.
I can sort of see the point Ivo makes about not being able to drop creating unit tests. But I think for that to be true you also have to have strict deployment rules that will disallow deployment without a certain amount of code coverage and all unit tests passing.
Which i guess ties in with what Stefan says about that many enterprise projects will even require TDD or at least a certain amount of code coverage.
Um… wait a second. You said that TDD sucked for *technical* reasons, but everything you offered for rationale was environmental. Customers not knowing the benefits of TDD isn’t a technical flaw; it’s poor salesmanship. That doesn’t weaken TDD, it illustrates ignorance on the part of the seller or buyer.
You said that a study showed TDD added development time to projects, which is fair… but you didn’t point out that “initial development time” is a poor indicator of overall project time. It’s only the initial development time.
TDD can be slower according to the WISCY principle (the “Why isn’t someone coding yet?” question from stakeholders who want to see immediate output from a project.) However, looking at it from almost any other perspective yields a different result: the code that comes *from* a TDD environment is usually more trusted, better documented (because the tests constitute valid use cases), and is of better quality overall (because building testability in generally gives you a better design.)
I’d say your argument and statement are flawed: your argument is that TDD is hard to sell, but that being hard to sell makes it a poor *technical* approach. Bzzt.
@joseph ottinger
umm, I don’t exactly remember stating that it was for technical reasons. I think I even make it quite clear that I do think that TDD results in a better product.
The entire reason for this blog post, is that I think most blogs about TDD focus too much on the technical benefits and too little on what you call the environmental effects.
Those environmental effects are very important when looking at the business case for TDD. Most companies don’t live in a academic clean room where everything happens by the book.
The reason I might have overstated some things and left out others is simply to get people like yourself to comment and address the very real business issues that often seem to be overlooked or not addressed TDD.
I understand that you overstated some things just to attract comments – but I find that to be a poor approach.
As far as a technical approach, here’s what you said: “For anyone who now still thinks TDD is a sound development methodology, I would be very interested to hear you thoughts.”
A *sound methodology* would be sound *regardless* of how marketable it was. I’m not debating how important the environment is; that’s a neutral claim on your part, and I can certainly see your point.
But to indict TDD as a whole, based on factors that are external to it? Ick.
That doesn’t mean that TDD is always appropriate for every circumstance, or that it’s an easy win every time – there’s a cost associated with it. Occasionally that cost is too high (which is what you are pointing out). But that’s definitely got nothing to do with whether it “sucks” or whether it’s a “sound methodology.”
@joseph ottinger
hmm, yes. ‘sound development methodology’ was on retrospect perhaps a wrong choice of words. I can see your problem with that.
I think that TDD coupled at a minimum with a good iterative project plan, that allows the customer to have a “valuable” product delivered, even if it isn’t the entire product that he might have wanted at the beginning will usually more than make up for the 15-30 percent overhead of TDD.
Let’s be serious here. Everyone has heard about the massive percentage of software projects that fail. Well, that is probably a bit exaggerated but there is some truth to it. The software development world is replete with unqualified opportunistic dilettantes. No one ever takes stock of this basic fact. I can’t count the number of so-called techies that have told me something along the lines of “I don’t need to know that to get this done”. Yes you do. All knowledge that leads to a more fuller appreciation of computing in general is good.
The reason for this is there is a glut in the market. There are too many darn computing jobs. So as managers and business analysts take stock of the total chaos of the software products being produced they start scratching their heads. As well, the dilettantes start scratching their heads and try to come up with rationalizations as to why their code sucks. Without ever even thinking for a second about cleaning up their code and being less sloppy they start making a bunch of tests. And these tests are generally the same as their code, resembling a small child’s messy bedroom.
It will always be the case that there exist a programmer who doesn’t follow TDD, produces less bugs, and produces more working code than does someone who writes a bunch of test. This may not always be the case but it is frequent enough for me to conclude that TDD is unnecessary. What is necessary is for so-called programmer out there to wake up and stop being so damn sloppy. Stick to the fundamentals and you will be ok. Get your head out of the clouds and for heaven’s sake use proper indentation. I may sound frustrated but this is where the current state of software development is. I simply can’t believe so many people are producing so much crap and the TDD conversation continues.
To many introductions to introductions being read out there.
I can’t agree more. I don’t think TDD is practical. Client can’t see it and employer doesn’t value it. I don’t give a shit too, because that is not a creative way, it treats people like machine. And too much test cases distract me from creating better features. And in most case I would say, with my 10 years IT experience, most codes and modules are not worth to do TDD, and most bugs can’t be avoided by using TDD. Pure waste of time in most case if you are a professional consultant. Of course some managers would like to brag it based on a different reason. But the people who makes things done and creates scalable system won’t buy it.
Well, if you want to sell unfinished software then TDD as well as agile development suck because they try to deliver finished software items. If you want to deliver a software full of bugs and your customer is willing to pay extra to fix those bugs (or you like to gamble and hope the customer won’t notice?) then why using things like TDD??
Your article is bang on the money in terms of client and business value, but off the mark otherwise.
Not only is TDD a waste of the client’s time and money, it is a waste of the developer’s time as well.
The software testing religion is silly. This idea that you should spend countless hours writing tests so you can boldly refactor all your code safe in the knowledge that you will never introduce a single bug due to your awesome tests violates every theory of computer science there is.
For one, refactoring the code often means refactoring the tests due to changes in the API or architecture. Changing the test when you change your code is equivalent to having no test at all. Read that again if you don’t understand.
Secondly, due to the halting problem, it is impossible to write tests that will find every single possible bug. If you think otherwise, then you should submit a scientific paper with your findings because you will receive the Nobel Prize and have the distinction of proving the likes of Turing and Djikstra wrong. More likely, unit tests exist solely to give you a false sense of security about your code.
Thirdly, when the project nears completion and you fix a lot of small bugs, you will often find you spend as much time maintaining the unit tests as you do the code! You just don’t have time to do that anymore. More likely, the deadline will force you to throw out all the unit tests because they aren’t adding any value and are slowing you down too much. In which case, you’d have been better off not wasting the time writing them in the first place!
And yes, unit tests can have bugs in them. Why do people assume that developers can write test code that is completely 100% bug free whilst assuming that those same develepers are incompetent enough to include the sort of bugs that unit tests will find in the main code?
It just doesn’t make sense.
In very specific circumstances, such as writing complex scientific or mathematical code, tests make sense ONLY for the purposes of regression testing. It is a complete waste of time in every other circumstance.
Good software development is efficient. Clients expect value for money, and if they knew their developers were squandering thousands of hours of their time writing things which add no value whatsoever they would probably sue.
There’s a good reason the market favours developer groups who don’t waste their time writing tests. They do the job more efficiently.
I used to work for DEC in the eighties developing and testing world-wide apps and now develop Web apps with .NET. I develop and test according to basic techniques learned there., namely, I have a spec that has been reviewed and that lists what a component should do or look like. I code it so it satisfies ALL these requirements and then tick them off as I test it. If I don’t get a spec, I write one that lists the requirements up front, make sure my boss reads it and deal with any issues and then code it so that it satisfies all requirements etc…. Test Driven Development …no thanks. Functional spec driven development plus testing ….yes. Also, of course, having a decent architecture/framwork that allows easy maintenance and extension also helps.
@Gary S
It’s good to know there’s at least one sane person on the internet. I agree that this whole test driven development religion is completely overblown and is a failed attempt to fix the reality that there are people out there that are just bad at programming.
Instead of actually knowing what you are doing, you write a bunch of tests and then mindlessly assume all your code is correct when your tests pass, not knowing full well that you might’ve coded your tests wrong in the first place. What will you do then? Suggest that tests be written for the tests?
So instead of wasting time writing tests to claim that I have “80% code coverage”, how about training software engineers to actually think, code, test, and clean up after themselves. Now all we have instead is a bunch of banshees telling me how to do my job and waste my time writing tests so they can layoff the QA department.