A Slack conversation brought up the question of why I do … the things that I do.
The inimitable GeePawHill has been musing on the practices that we—how shall I say—agilist programmers all seem to share. He’s trying out new ways of saying the same old things, since we’ve all been saying the same old things, in different ways, for over two decades. Our interest, all of us, is in managing to say the thing that will capture the thoughts of another new tranche of people, who will thus benefit and who will then help us learn and advance the ideas we share.
Finding new ways to say old things is quite valuable, I think, because no two of us mean the same thing by the same words, nor hear the same words exactly the same, so another way of expressing what we think, believe, or feel, is quite valuable in reaching more listeners.
GeePaw mentioned his new phrasing on a Slack we both belong to, and a number of us chimed in with comments, germane and otherwise. In the course of that conversation, GeePaw asked us all:
Why do you do TDD?
I propose to answer here, a slightly larger question:
Why do you do what you do?
To the extent that we know our reasons at all, they aren’t likely a straightforward list, I do this to get that which gives me these so that I can have those. Our reasons are a tangle of forces and responses, by their nature often hidden, and almost never able to be isolated down to “always do this”, or even “in this situation always do this”.
As a consequence, what I write here will be wrong when it sounds direct, and confusing when it begins to reflect what I think I really think. With that warning, let’s begin.
What do you do, anyway? And why?
The primary lesson of “Agile”, to me, has been the notion of “working software”, what Scrum calls “the increment”. The Manifesto says things like “continuous delivery of valuable software” and “working software is the primary measure of progress”. As I look back on my many failures in the software profession, the bulk of them seem to fall into a single classification of causes:
When time ran out, we did not have a working viable product to deliver.
Sometimes the time that ran out was a justifiable date: to deliver tax preparation software for a given year you simply must have it available for the tax professionals’ conference the year before. That’s where the preparers assess products and decide what to buy.
Sometimes the date was arbitrary, with no connection to any reality except one: when the date had been sufficiently exceeded, and no software was forthcoming, management made another seemingly arbitrary decision to cancel you, or the project, or both.
Sometimes the date was unknown. When the capital ran out in a company whose management was on the shady-silent side of things, they just closed the doors if there was no software to be had.
The common pattern is my life was: when the time runs out and you don’t have a product, bad stuff happens. (And you don’t even know for sure when the time will run out.) The solution I learned was:
Always have a working version of the product ready to go.
Now in 1996, I had experienced the problems above, but what I didn’t know was that it is possible to have the solution. As I learned “Agile” ideas and practices, I learned that we can have a working product, practically from day one on.
Let’s talk about how to do that, and why.
The why starts out pretty simple: if we’ve got the product, they can’t say “you didn’t give me the thing by the date” quite so easily. They might quibble over features, but if we can make the damn thing work every day, we can certainly put in the features in the order of most important first, so the whole conversation is changed from “you didn’t do X”, to “do X next”.
Of course, having even the most important six things done won’t always triumph. They might legitimately need seven or seventeen. But it will change everything. I’ve written about that before. Here, let’s just say that I want a continually growing working product, my reasons are roughly as above, let’s see what that implies.
It has to have value
To have value, the software has to actually do useful things, and it has to work correctly. The more useful things we can correctly add in as time goes on, the more value it has, and the better chance it has of being valuable enough to keep the effort alive.
Therefore I do these things:
- I try to choose features in order of importance;
- I try to make sure that everything always works;
- I keep it working as a product not just as pieces.
Choosing what to do next needs to please the people with the power. It’s best if we let them choose. It’s good if we find out what they want and choose it. It’s not so good if we just guess, or do things in some “technical” order.
This gives us practices like XP Customer, Scrum Product Owner, and good old management-technical communication.
Making everything always work requires testing. Every time we change or add anything, there’s a chance that the new thing doesn’t work, and a chance that some old thing doesn’t work. Not too long ago, in my Space Invaders program, the brand new flying saucer didn’t fly if the left-most bottom-most invader had been killed. The better my testing is, the more likely I am to avoid shipping software with problems like that.
This gives us practices like automated testing, test-driven development, acceptance test-driven development, and making testing part of the process, not some kind of tail on the process.
Making it work as a product requires me to integrate the work of all the people involved in the development. If it has to work “all the time”, their work has to be integrated “all the time”.
This gives us practices like continuous integration, and leads to practices like continuous delivery.
We could elaborate the reasoning above further, and draw out many more practices and approaches, all aimed at the continuous delivery of working software. I’ve written on that in the past, and surely will do so again in the future, given certain assumptions about the deity and the flood plain.
I want to talk now about how I work day to day, moment to moment.
It’s working software all the way down
As I write software, which I do almost daily, I’m always trying to keep my product working, while growing it in capability and internal quality. I try to make it better every day. Since I really only work on something like a Space Invaders version and article for a few hours, I try to make it better every couple of hours.
In fact, I try to make it better every twenty minutes or more frequently. I choose twenty minutes, because I’m pretty sure that if I haven’t made progress in twenty minutes, I should put down my coding pencil, take a break, and start over. Often, I don’t do that, instead banging my head on the desk and poking myself with the pencil for an hour or more. And then I take a break and start over. Rarely, I manage to dig out without starting over.
So what do I do that lets me make progress every twenty minutes?
First, I pick a very small thing to accomplish. How small? Smaller than you’re thinking. As small as possible. An early thing to accomplish with Space Invaders was to display a block on the screen, representing an invader. Very small things.
Then there’s a branch. Sometimes—the best times—I begin with a test, doing the thing we call TDD: test-driven design. I write a small automated test, showing that the Very Small thing I’m going to create doesn’t work. I run the test to see that in fact the VST doesn’t work. Then I write the least code I can imagine that makes the VST work. I run the test, showing that it works. And in fact I run all my tests, showing that the VST works, and that nothing else is broken (as far as my tests know).
Then I pick another Very Small Thing.
Let me drill down to how small a VST can be, just for fun. There’s an idea called “fake it till you make it” that is part of TDD thinking as Kent Beck defined it. Imagine that we’re creating a feature that will return the transaction summary on a user’s account. We choose as our first Very Small Thing, an account with no transactions, we write a test that checks to be sure we get zero. And then to implement the feature, we write whatever code we need that finally calls the summing function that we’ll have to write … and instead of writing the function, we just return zero!
Why is this not a Very Stupid Thing to do? Usually when we create the very first test for an object, there’ll be a fair amount of code to write or generate. Class definitions, method signatures, whatever all our language requires. We will quite likely need to produce more lines of code for that initial implementation than we ever will do again in a single batch. So we make it as short as possible, with “return zero”. We know we’ll be back soon with an account that does have transactions, and we fill in the summing part then.
So often, but not always, when I create the first test for a Very Small Thing, I use this fake it till you make it trick. Why?
It shortens the time when my tests don’t all run!
When my tests don’t run, I don’t have a product, I have a broken thing. When they do run, everything I’ve set out to do so far, from day zero until now, works. It works! I don’t like it when the program doesn’t work; I like it when it does work. So I move into Not Working and back out as rapidly as I can.
At my best, I do this Very Small Thing ritual using TDD, the practice of writing the next automated test, seeing it fail, making it work, and then making sure the code is good. Sometimes, I don’t use TDD. Sometimes I can’t quite see how to do it. How could I write an automated test to show me what was on the screen when I displayed that block? I don’t know of any way that my program can see the screen. So I didn’t TDD that.
But I did do a small thing that didn’t work and then make it work. I did it manually, which gave me the same rapid transition from not working to working, but it cost me the reassurance that an automated test would have that it’s still working tomorrow.
So if I ask myself:
Why do you use TDD?
I’d have to say that I do it because it gets me from Working, through Not Working, and back out to Working, as rapidly as possible, while showing me that everything that has gone before is (probably) also Working.
Why do I do that? I do that because I want to have a working product all the time, containing the most important features so far.
Why do I do that? Because I’m afraid of what happens when time runs out and I don’t have a product.
But I have other reasons, reasons that apply even when I’m not afraid that I might fire myself, which so far I have never done.
In using TDD, I’ve discovered some things that offer me value:
- I tend always to have a working product
- I have more confidence in the product
- I am more relaxed working with the product
- I get lots of little jolts of success from my small steps
- I get stuck or confused less often than without TDD
- My code becomes simpler
- I get more done
That last one: I get more done. Am I scientifically certain that I get more done when I write tests than when I do not? No. I have not done a series of double-blind experiments on myself, nor do I think it’s possible to do so. But am I certain that I go faster? Yes, I am. The reason is that when I TDD, I very rarely have to do any serious debugging, very rarely find myself confused for hour on end. Not never, just less often.
And when you’re doing a feature every twenty minutes, it doesn’t take very many hours of confusion to leave you short of a few dozen features.
This is an example of what GeePaw has called “The Money Premise”, that we do TDD because it lets us build more software for the money than if we didn’t. He believes that’s true, and so do I.
So yes, I go faster to a working product when I use TDD than when I do not. But that’s not really why I do it. I’m not getting paid for what I do. When’s the last time you sent me a paycheck? (I’ve been waiting …) Even when programming for money, I don’t likely get paid more for working this way, although I might get paid longer if I manage to ship enough value in a short enough time. Even so, I’m not sure the Money Premise works for the programmer. I am sure it works for management and the company, and I might express its value to those people that way.
As a programmer, I do TDD because I get more jolts of success, because my code is cleaner, because I experience less stress and more ease. I do it because I like how working feels when I do it. It kind of feels like … not working. If kind of feels like programming for the fun of it.
I’ve droned on long enough here, but let’s take a look at some other practices that I like and recommend.
Refactoring is the process of improving the design—the internal quality—of working code. It’s useful for at least two reasons. First, we often code something “quick and dirty” the first time around, because we know the algorithm and want to blurt it out as quickly as we can. Second, as our needs evolve, the code we have often needs to be improved to make space for the new needs. We could rip it all out and rewrite it, but that would take exponential time. Instead, we improve the code locally and globally, in small steps. We can even do the improved design over time, with new features interleaved in between design improvements.
I do that because when I don’t do it, I slow down as the design gets worse. At some point, the design gets so bad that I’m almost afraid to change it. I begin to paste things on the outside of it, making it even worse. I imagine that all big systems eventually get so crufty that they become impossible to maintain. Refactoring holds that day off for much longer.
Even more importantly, refactoring today literally makes tomorrow’s work go faster. Refactoring makes me go faster. Again the Money Premise.
But refactoring also gives me ease, and gives me more pride in my work. It benefits the customers and the company as well. It may or may not put more money in my pocket, but it definitely lets me enjoy my work more.
I like automated acceptance tests of functionality for a number of reasons. Since I can run them any time I want, they give me assurance that nothing is newly broken. But most important, they refine my agreement with the customer as to what we’re setting out to do. They turn general statements into specific examples, isolating exceptions and helping us be sure we are on the same page.
They make the process of developing the needed feature faster, because we make fewer mistakes and we can zero in on exactly what’s needed. Money premise again.
They help me produce value faster.
I’m going to stop now. If you want to know why I do some other thing that I do, feel free to drop me a line or tweet at me or something. I’ll try to explain, on Twitter or here or on the Agile Mentoring slack or some other medium.
But mostly, I do the things I do because they let me deliver value faster, and help me to develop software with a sense of ease as opposed to a sense of impending doom. There’s enough impending doom to go around without me creating more of it.
Build your software with ease … see you next time!