Another long-ass post on software estimates

I had intended to clarify my position on this excellent parable on software estimation months ago, but forgot about it. Then my co-worker Chris Nelson wrote a lovely piece and asked for comment, which I provided.

It, uh, proved to be too long for WordPress’ comments system. So I emailed it. To a DoNotReply@ address. Cascading failures. So I reprint my comment here.

Software is different from other forms of engineering in some fundamental ways.
For one, we rarely begin from a fully formed specification, a shared knowledge base and a known set of tools. Instead, we’re frequently called upon to use brand new tools in a brand new domain to meet a spec drawn on the back of a greasy paper plate from the pizza shop. In a road building analogy, you might begin without a clear understanding of whether you’re building a footpath, an elevated highway, or a runway, and you may have one expert in each of these on the team. And it’s entirely likely you’ll realize what you’re building is a subterranean parking garage.
It’s also different in that you can build partial systems and change them rapidly while they’re under use. Nobody would build a bridge that goes one way and takes traffic before all its pylons are secured, but in software we commonly do just that — show the facade of a fully constructed bridge and open traffic to a single lane, just to see if our bridge provides value to travelers. Maybe they wanted a tunnel. We can change that in the middle of a commute.
Finally, the velocity (work performance) of a given engineer can vary wildly from day to day and from task to task. Part of this is different expectations of what is to be built, part of it is how much of the problem is “in cache” (awareness of code, awareness of the construction process, awareness of the domain) and part of it is simply how people work. There are many software enterprises that heap accidental complexity on developers in the form of silly processes, bad tools or unwise technical requirements delivered by nontechnical employees.
There’s an equation I like to use wrt estimates:
delivery date = start date + (work / velocity)
In an industry with “blue book” values for how long work will take at a maximum and consistent minimum velocity across employees, you can have very reliable delivery dates. In software, we’re actually very good at identifying blue book style work — so good that the blue book value is 0. We turn the generation of trivial work into a scripted task.
This means that any remaining work is necessarily highly uncertain, and that uncertainty can blow up any of the three variables. It’s a coefficient, but not one you can solve by simply doubling estimates — in the extreme, the value of the coefficient of uncertainty approaches infinity.
And it’s very difficult to scale velocity — typically we solve this by adding new people to the team, but the incoherence penalty caused by bringing their velocity up to par degrades the total velocity of the team such that they are now further behind than they started. You can ask people to work longer hours, but the increased defect rate and decreased performance of overtime introduces another incoherence penalty after about 9-10 contiguous hours. You can improve velocity by organizing to optimize engineer performance — this can be a huge deal, read “the o ring theory of devops” — but you will eventually hit a limit, and velocity improvements take a long time to materialize.
So velocity should be considered more or less invariant. The only thing you can do is pick one of the other two variables and make them invariant, allowing the third to blow up. If you don’t do this, you should expect both work (including quality) and the delivery date to inflate.
Scrum and Kanban, in the abstract, attempts to solve this problem by setting a concrete deadline, but allowing the actual work to vary according to available velocity. This is where your single lane bridge is developed, a lane at a time.
And “Estimate-free” development is common when the work absolutely, positively MUST be done, regardless of schedule. In place of estimation, a bare-minimum work scope is defined and progress is regularly reviewed — but a release date is neither known nor announced until it becomes obvious to the team. (Generally there IS a deadline for this class of work, but it’s more a checkpoint date to verify the aptitude of the assumptions that drive the work, not an expectation of a working product).
In the above parable, engineers were delivered a due date, which became an estimate, and they began a work plan. As the work continued, the estimate was revised — but the work plan was not, even though it became obvious they could not meet the date. Human effort cannot make up the difference.
In a SCRUM version of the parable, engineers would start walking, walk for 1 day uninterrupted, and then report their progress back. A new plan would be developed — maybe, instead of walking, you take a bike. Or maybe, instead of walking the coastline, you follow the highway. After 1 more day, work would again be compared to the plan, and so forth. Eventually, everybody would realize what’s really important is that the team be in LA on the end date, and somebody would suggest they take a bus. The rest of the walk would be logged as technical debt, eventually abandoned and life moves on.
In an estimate-free version of the parable, the date doesn’t matter but the walking of the coastline does. After about a week, there would be a meeting to discuss progress and remind everybody exactly what matters about the walk — maybe seaside views matter but climbing every peak doesn’t, maybe we bypass part of the coast by hiking along the 101 from San Luis Obispo to Santa Barbara — and the team would go back to walking. Management would dispatch outfitters to keep them motivated and productive every day, and eventually they’d complete the route – but it’s going to take 20-30 days, dependent on uncontrollable externalities like the weather, injuries and trail quality.
By the way, it’s not possible to eliminate the unknown externalities that make estimation impossible but a good prototype, built on a tight time constraint, can help to identify them earlier. If you plan before you prototype, the plan’s pretty much guaranteed wrong.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s