The Testing Conundrum

When TDD was described by Kent Beck in his 2002 book Test Driven Development, software testing was in a highly dysfunctional state. At one extreme, small companies that maintained products generally handled testing in an entirely ad-hoc manner, leaving it to programmers to test their own code (or not). At the other extreme, large government software projects required detailed manual test plans to be created ahead of time.

Neither of these extremes worked very well. One problem was that tests and code could easily diverge. Another problem was that testing usually occurred during a test "phase", at the end of a development "phase". Of course, myriad problems were detected by the testing during the test phase, but software projects seldom allocated enough time to fix the problems. The result was that the test phase was a miserable pressure cooker in which developers had to labor overtime fixing things in order to meet a project deadline, for a project that assumed that testing only consists of testing but not fixing, let alone refactoring. The assumption - deeply flawed - was that the up-front design was perfect, and so any problems found during the test phase would be trivial ones that are quick to fix. Dream on!

So when TDD arrived, it was a solution because it addressed the problem of keeping tests and code in sync, and it prevented the testing crunch period. It also encouraged teams to measure test coverage.

The assumption that an up front design will be correct, which is almost never the case, was known to be false for decades before TDD came about. But the trend had been toward software “procurements” that separated design and development into distinct phases, which did not work because it was purely top-down – an extreme. Separating work into phases without iteration prevented learning and the process of refinement - essential for designing and developing correct software.

TDD was a clever and oppositional counter to the exclusively top-down design approach. The problem is that TDD is also an extreme, and is unsuited to how many people think (more on that in the section Why so many people don't like TDD). TDD tends to sideline design, and – contrary to claims of TDD proponents – tends to result in an insufficiently tested system (more on why later).

Before I critique TDD, let me first say that it defines a foundational model for Agile development: the red-green cycle, which is a feedback loop that ensures that tests exist and are maintained over time. In defining the red-green cycle, TDD has made a huge and critical contribution to software engineering; the problem is that it also defines a granular process that is often not effective.

TDD, by being bottom-up, achieves design through the process of refactoring. The idea is that design is “emergent”. That design “emerges” is a myth and an oxymoron, because design is, by definition, intentional. What emerges is structure - not design. Look at any old city and you will see how poorly organized an emergent structure is. And the giveaway that humans are not the product of intelligent design is that the human body has very poor design “cohesion”, and that is because we are the result of evolution – not intentional design. It has lots of structure, but changing anything has untold countless side effects, and the very narrow range of conditions that humans need for survival is the result of the fact that we are like a brew that grew and that can die if anything changes just a little. A robust design would not be so sensitive and so strangely interconnected.

But let us remind ourselves that testing is not the goal: assurance is the goal. Agile+DevOps is the ability to release product features rapidly – but with high assurance.