Programming can't be successful without design. But I begin my projects with very little design and seem never to do any. On the contrary, I'm designing all the time. That's what makes incremental delivery possible.

No Design? No! All Design!

My esteemed new interlocutor, Jonathan Pierce, has a very strong up-front design focus, and has trouble seeing what I’m up to in this series of articles. That tells me that I need to be more clear about what I’m up to. So I’ll digress for a bit here, in case any other readers exist – and are having the same issues.

It’s commonly thought that the best way to produce good software is to carefully analyze the requirements up front, then carefully design the data structures and processing up front, then code up the design, and then test the system and ship it. What’s most interesting about this guidance is that when, as often happens, our projects don’t come in on time, on budget, fit for purpose, we conclude that there would have been less change if only we had done more work on the requirements, that the design problems we had would have been lessened had we only done more design up front, and that if we had been more careful coding, the testing phase wouldn’t have taken so long. The worse it works, the harder we try to make it work.

Remember that old definition of insanity: doing the same thing over and over again, expecting different results.

Well, as you probably know, agile methods and XP recommend approaching projects in a very different way. We’ll take a quick review here of how I think about it.

Value of Incremental Delivery

Agile and XP start with incremental delivery of the product. This is an incredibly valuable practice. Here’s why:

  • If we can actually deliver part of the product earlier, our return on investment is higher. All the time and money spent before we put the software into use is investment: it has no value to us until we turn that investment into usage. The sooner we ship truly useful software, the better we do.
  • Even if the software cannot be put into actual use, we get information from having it ready for deployment. We can test it, our customers can touch and feel it, and often discover that what they asked for wasn't what they meant.
  • If we estimate the difficulty of features, and observe how long it takes to implement some of them, we can begin to predict very accurately how long it will take to get done.
  • If the time and money needed to do whatever our customers want is less than they want to pay -- and when isn't it -- the sooner we find out, the sooner we can make the best possible decisions about which features to do before the deadline, and which to defer.

The bottom line is that if incremental delivery is possible, the payoff is great in terms of return on investment, learning, information about progress, and the ability to steer the project. Do the math. Even if incremental delivery costs more, it might well pay off. And my experience is that it doesn’t cost more – it costs less.

Requirements to Begin Incremental Delivery

OK, suppose we wanted to ship early and often, because we want all those benefits we just looked at. What would we have to do?

  • We need to go through a very rapid requirements phase. The longer we wait, the longer we put off those benefits. We would like to identify some key features, and get started right away.
  • We need to spend very little time in up front design, for the same reasons: the longer we wait, the slower our return on investment.
  • We need to spend very little time building infrastructure before we start getting features. Again, the sooner we start, the sooner we collect the benefits.

Putting this all together, if we are to gain the benefits of incremental delivery, we need to start writing software ASAP, with a very simple design and very little infrastructure.

Requirements to Sustain Incremental Delivery

Now, it’s easy to see that we can take a few requirements and code them up with very little up front design and with little or no strong infrastructure. But we have all seen the results of too little design and too little infrastructure. That leads to projects that slow down, slow down, and finally end up dead in the water. If it’s possible to keep the progress steady, there are things we’ll have to do:

  • At every moment in time, we'll have to have a design that's clean enough to move forward. That will require continuous design improvement, if our project is to be possible at all.
  • At every moment in time, we'll have to have infrastructure that's good enough to support the current functionality. That will require continuous building of more and more infrastructure.
  • We'll need to be able to evolve the design as we go, without breaking the existing functionality. That will require comprehensive automated tests. Anything less would introduce problems and slow us down more and more.

To sustain incremental delivery, we have to design all the time. To someone steeped in the notion of design as an activity done up front, this can look like doing no design. It looks the other way around to me. Those folks design a bit at the front, maybe as much as 25 percent of the project. Agile developers design 100 percent of the time! (See References)

Summary of Agile Requirements

To do a project in as incremental a fashion as possible, “all we have to do” is get started as soon as we know a few important requirements, create a simple program that delivers those benefits, but that has a good enough design to allow the next feature to go in readily. Then “all we have to do” is change the design and infrastructure as we go, using a comprehensive suite of automated tests to make sure that the system continues to work.

As long as we can do those simple things, at a reasonable cost, we can build and ship our product incrementally, gaining the benefits of early and frequent shipment.

My Work Here is Done, er, Just Begun

My book, Extreme Programming Adventures in C# is all about how to do incremental development of software, with the added issue of the programmers in question not being very experienced with C#. And this new series, Discovering Better Code, is addressing the same topic, at a somewhat different scale.

In this series, we’ll address smaller projects, but always with an eye to the same issues.

  • How do we start simply and keep going rather than bog down?
  • What do we need to test, and how can we do it?
  • When our tests let us down, what can we do about it?
  • How do we recognize areas of the design that need improvement?
  • How do we improve the design in small steps, avoiding expensive rewrites?

As you read these articles, you’ll see what really happens as we work on these programs. You’ll hear us thinking about the code, see what we do, what we learn, what we do about our mistakes. This ain’t no cleaned-up, dressed-up, idealized story about perfect examples of perfect design. This is down and dirty, how do people who are reasonably expert in incremental development think about and do their job.

My hope is that by watching what we do here, you’ll learn some techniques that are useful to you in your work. If you do, you too can start to get the benefits of shipping early and shipping often.