Estimating Software Projects:

Software Estimation Is Hard. Do It Anyway.

It’s well-established that estimating software projects is hard. One study by HBR found that one in six IT projects had cost overruns of over 200% and were late by almost 70%. Another study by McKinsey found that IT projects are on average 45% over budget and 7% over schedule. They found large software projects were particularly bad: software projects with budgets over $15M went over budget by an overage of 66% and had schedule overruns averaging 33%.

Of course, anyone who’s worked in software for a bit will have seen this first hand. At some point you’ve probably said “oh yeah, that’ll only take a couple of days” … and then found yourself, a month later, still not finished. Estimating a software project seems to always run up against Hofstadter’s Law: “It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Unfortunately, it’s common to look at this pattern, see that estimating software project timelines is hard and just … give up. There’s an established “No Estimates” position that says we should entirely stop giving estimates on software projects. Many Agile methodologies involve arbitrary scoring systems – story points, t-shirt sizing, etc. – deliberately designed to help avoid giving estimates in time-scale units.

There’s nuance here: these ideas do have some value. I don’t mean to dunk on story points. In particular, “no estimates”-style project management can lead to better conversations about timelines: these systems encourage questions like “what can we get done in the next two weeks?” instead of “how long will Feature X take?” This can often be a much better way of thinking about software projects, especially long-lived ones.

However, sooner or later, someone’s going to ask “when will Feature X ship?” There are situations where having an answer – an accurate one – is non-negotiable. Maybe Sales can close a major deal if they commit to a timeline for some new feature. Maybe the feature in question is a major dependency for some other team, and they need to know how to schedule their work. Maybe your feature is one of many that will come together into a new product launch, and the whole product launch apparatus needs to spin up and promote it, which requires a timeline. I could go on: the point is, there are many situations where an estimate is required.

One major “secret” to advancing in a technical career is learning how to give accurate estimates. It certainly has been for me: I don’t shy away from giving timelines, and I’ve learned how to be right often enough that folks trust my estimates.

If you always avoid estimation and don’t learn how to give a timeline when it’s required, that might become a limiter on your career. Being able to tell your bosses and peers what to expect by when – and then hitting those marks – builds trust in a major way. This is true of individual contributors and engineering managers: you can pretty easily become a Senior Engineer or Engineering Manager without needing to be good at estimates, but to go much farther than that you’ll likely need to learn this skill.

And it is a skill: estimation can be learned. You can learn techniques, but to some degree, they don’t matter: whatever technique you use, practice makes perfect. If you get in the habit of breaking projects down and making timeline estimates, you can review those estimates against reality and recalibrate. Do this repeatedly and you’ll get better: you’ll learn where your team tends to get stuck and where they blaze through work; you’ll find which areas of your codebase are quick and easy to modify and which require more time; you’ll learn to recognize your own biases and where you’re most likely to over- or under-estimate.

The next post in this series covers the technique I use for estimation, one that’s been effective for me. But the technique doesn’t matter as much as the broader point: estimation is hard; do it anyway.