We’ve been working on the XProgramming replacement project since late September, so let’s talk about estimation.
Estimation is important
Project cost estimation is important — at least it is to me. I wanted to get XProgramming.com off of WordPress, but not at any cost, no matter how high. At some point, it would make more sense to learn more about maintaining it myself, or to hire a new contractor to do it for me. Since there isn’t much maintenance to do, it would probably only cost $100 per month or less.
OK, well, $100 for how long? Ten years? I might live for ten more years, and if I do, I hope I’m still writing stuff for people to read. Both of you. So that would be $12,000 in current dollars. Yeah, but that’s a big cash outlay, so my cash limit was surely lower than that.
So I felt an intense desire to know how much this was going to cost, in money and time. I wanted to be sure it was possible in the time and money I could invest in it, because if it wasn’t, I wouldn’t start the project at all.
I had a gut feel for the budget: I was thinking it would be worth doing if it was a few thousand dollars and a few months. Few? In my mind, probably three.
Ask an expert
My colleagues in the #NoEstimates movement have suggested that to get the estimate, I should find someone who is in the business of converting WordPress to static and ask them. Let’s count the reasons why that’s not a good idea.
I don’t want a static version of a WordPress site. If you don’t know the internal layout of a WordPress site converted to static, count yourself lucky. Inside it’s about as beautiful as those animals you see sleeping by the side of the road.
I want a particular approach to writing articles. I want to write in Markdown, with each article in its own folder, with the articles assets in the same folder. By assets I mean pictures, attached PDFs, whatever.
I want to be able to maintain the site from now on. This has at least two implications: it needs to be very simple, and I need to be engaged in every aspect of building the site.
I could go on, but this is enough. This is an “in-house” project. We might bring in some help, like the kid across the street, but it’s going to be done in my garage, using my dad’s tools, out of the scrap wood we can scrounge around the neighborhood.
Just like most everyone’s projects
Now this is important, not because it’s me, but because so many projects in business are quite similar. They’re going to be done “in-house”, by existing people, who have other things they have to do, and other things they might be assigned to.
There’s some rough budget limitation in money and time. It might be really solid, it might be pretty vague.
The benefits of the project are hard to quantify. Someone wants it. They think it’ll make their work go more easily. They’re willing to throw a bit of time and money at it. In business, the numbers may be larger, but the problem is the same.
Someone really wants to know how much and how long. They won’t die if they can’t find out, but they are not going to be happy.
Think about that. I understand the incremental, iterative approach. I was there when the Agile Manifesto was being written, remember? I’m in the picture! I get this stuff.
And I did not know how long it would take, and I knew my time and money were at risk of being lost entirely, and I did not like that. If I were not such a reasonable boss, I’d have fired myself for not knowing how long it would take.
When we’re pushing back against cost estimates, we often suggest looking at the value of the project. Presumably if the value is high enough … what? What are we going to do if the value is high enough? Go ahead willy-nilly? Hell, no! If this is actually important, we’re going to want to nail down the costs more solidly. We need to be sure that we invest enough to get all that value in time.
Frankly, I think asking someone who wants a cost estimate to think about value instead is obviously a diversionary tactic. Even if the individual agrees that value is interesting to talk about, they still need to know how much and how long.
Another approach to avoiding estimates is to set a budget. What? How are we supposed to come up with this budget, pull it out of the nearest orifice? Now we may be able to set limits via budgeting, and in my opinion we should. But that doesn’t change the fact that to set the budget we need to know how much it will cost, in time or money, to get a decent version of what we want.
Are you with me so far?
We got asked how much it would cost and how long it would take to do something. So we ask the boss “Well, how much is it worth?” The answer might be “It’s worth millions, but you can’t have it”, or “It’s not worth a dime, but it’ll get Jack off my back”. Either way, the value exploration may be interesting but it doesn’t answer the question.
Then we ask the boss “OK, well, what’s the budget?” The likely answer is “Hello, Junior. How stupid do I look? That’s what I’m asking you!”
The Value assessment and budgeting responses are both great things to do, but they do not satisfy the boss. And depending who the boss is, they could anger him, or cost us the business if he’s looking for a contractor.
Well, break it down
It’s common, when we can’t guess the whole number, to break the project down — and then guess the smaller numbers.
This works, to a degree, sometimes. Mostly it just gives us five or twenty things that we can’t estimate instead of one, but they are smaller, and some of them we might actually have a clue about.
The best thing about a detailed cost breakdown is that you can pad each individual estimate, which gives us a lot of padding to move around when it turns out, as it inevitably will, that our estimates were hogwash, drivel, bunkum, poppycock, and all the synonyms for what we all know estimates are.
But if you’re against cost estimation, and I certainly am, breaking down one estimate into a whole passel of things to estimate is self-defeating. It might work, and my motto is “if it works, do it”, but a more adamant #NoEstimates person would resist.
Well, try it
Now here we are with what seems to be the standard position for some #NoEstimates folks, and the fall-back position for others: “Let’s do some work and see what happens.”
This is in no way a bad idea. Unless we already know how long and how much, the best way to find out much about it is to start doing work.
Even if you’re going to use the “break it down” way of getting estimates, doing actual work is an incredibly powerful way to proceed. By doing the work, if you approach it wisely, you’ll discover things in the breakdown that you’d never have though of in a paper exercise, and you’ll understand them better.
We might have figured out just by thinking that there are about twenty substitutions that have to be made to convert WordPress HTML back to Markdown. Maybe. We might have figured out that most of them could be done by a “simple” substitution in the text. I’m pretty sure we’d have underestimated how long each one would take in real time, getting it right and being sure.
We might have figured out how tricky it would be to get the right articles selected for the highlight sections of the home page. We might have had some ball-park estimate for the CSS to make the site pretty, and responsive.
Frankly, I don’t think we would have figured those things out without doing some work. I was there, and I have been surprised by the realities of these issues even after talking about them.
The advice to “try it” is very good advice. And it can be a decent move in the estimation game, because you can answer the question “how much and how long” with something like “it will take me three people for three weeks to answer that question”.
There might be some push back, but you can reply that by that time you’ll have a more solid answer, and at least a prototype of the software for people to look at. “Try it” is a solid strategy if you can make it happen.
A very strong move if you can “try it” is to ship something. The point of my project is to convert XProgramming.com. The original plan was to convert it in place, at the same URL. However, that has changed a bit.
As we began to write code that produced a sort of web-site looking thing, we tested it locally, but soon began to want to see it on the web. We toyed with a number of approaches, including Heroku and GitHub. That consideration had some real value.
Bill Tozier, my main partner in this project, knows all kinds of tools and approaches to doing this stuff. It’s his hobby or business or something. Suffice to say, he knows a lot. Bill is into the technology. He has that technologist way of saying “Just put it up on Heroku” and making it sound easy. And I’m enough of a technologist to be interested.
My “Agile” experience causes me to move fairly quickly to trying things, or at least to digging in enough to see what trying them will take, and those considerations have led to a refinement of the project goals. Fundamentally, they’ve caused me to realize that I no more want to maintain a Heroku site than I do a Wordpress one. I don’t want to be reliant on GitHub and all their works and pomps.
All this has led us to a simpler and simpler approach. And as the approach got simpler and our examples got closer to working, I realized that I own ronjeffries.com and we could put the new stuff up there. And we did.
We have a product! It’s alive!
The impact of this on the project is quite high. It keeps us going, because we can see a result with actual end-user value. Not much (o! faithful reader) but some. In addition, it has refined our understanding of what we might do.
Just keep going
I think some of my #NoEstimates colleagues have in mind that, after the three people for three weeks thing, you’ll just keep going until the prototype is solid enough to be the real product. Maybe, sometimes, that will even happen.
But I’m not convinced. Even if the project is going well, there are opportunity costs to consider. The team is working on the XProgramming Conversion Project. They could be working on something else: a new book or the full-color vellum edition of The Nature of Software Development. There’s other stuff we could be doing.
When is this conversion going to be done? That’s a valid question, because it is consuming a big chunk of my productive work time. Note, however, that because ronjeffries.com is up, we have ensured that we aren’t left with nothing.
As it happens, we’re very close, because an invisible part of the project has been in converting the old articles to the new format, and that’s quite close to done. (For values of “quite close”.) The main thing holding me back from pushing everything to ronjeffries.com is indexing, which is part of what we’re working on now.
So yeah, sure, we got talked into “Just try it” and now we are in “Just keep going”. That’s a good thing, isn’t it? Isn’t it?
Good question. No answer.
Well, if you’re a programmer working for a living, yeah, it’s a good thing. You got me to go away and settle for a non-answer to how long and how much. You got what appears right now to be lifetime employment building my web site.
But that’s not a sustainable situation. I will time out on this. I will decide that I’ve spent enough money. Right now, were I to decide that, I’d not have enough. Pretty soon, with a little formatting and indexing, I’ll have a decent new site. I didn’t want a new site, but I kind of like it.
How long until I can move all of XProgramming to this new technology?
I still do not know. I believe that if we continued to work at the present pace, probably before 2015. That’s not as bad as it might be: today is November 24, 2014. Given realities, including the fact that there are never any working days in December, and that Nature is in final copy-edit, and that the Scrum Alliance needs my attention, I figure January.
If I were going to be held to it, I’d say “First quarter 2015”.
That’s it. Working for two months, I’m fairly willing to state with near certainty, almost, that it will probably be pretty much done by six months after we started.
That’s not very good. But it’s the best I’ve got, and the best most of us have. There’s some guy in Colorado who can do better, bringing in multi-zillion dollar projects on time and on budget. Yay, Glen. I have no doubt he can do it. And I don’t have much doubt that I cannot, and that most of us cannot.
Maybe we could. And maybe our six month couple of guys project would turn into a multi-zillion dollar effort. That would be good if someone else was paying.
This one, I’m paying for. I’ll stick with this approach. But darn it, I really wish I knew a better way.