Wednesday, 21 January 2009

Single point estimates are meaningless

How useful is statement "We estimated that project X will take 6 months"? What does it exactly tell you? Does it mean that you can be sure that the project will take 6 months? It's obvious that you can't be sure of that ... so if you were to sign a contract what would you do? It's quite typical approach to add 10% or 20% "just in case" which, in normal thinking, should guarantee meeting a deadline right? The problem is that single point estimates like 6 months for project X are useless without information about probability of success. Let me explain what I mean by that ...

Estimates are based on assumptions, specification etc. ... general rule is that estimates can be as accurate as details you have. So if you have only vague idea about the system then due to uncertainty your estimations can be really far away from the actual time. Some books are saying that actual time in such cases can be 4 times bigger the the estimate but it is also possible that it can be 4 times smaller! In our case, it means that the project X estimated for 6 months can take 1,5 to 24 months! Horrifying, isn't it?

Precision of estimates can be improved from ±4x to ±1,25x by providing details, that is why specification (requirements, UI design, use cases, acceptance criteria) can be so helpful.

What is the conclusion?

A good estimate is an estimate that provides a clear enough view of the project reality to allow the project leadership to make good decisions about how to control the project to hit its targets‥

Single point estimate doesn't provide a clear view, I believe that to have a clear picture it's required to know the best case and the worst case estimates. Then probability of success can be illustrated like this:

In the best case project will be finished in 20 weeks, but there are only 10% chances for that. In the worst case project will take 32 weeks, but also after week 28 we have 90% chances to get everything done. With those data project manager has a clear picture and can decided what risk is willing to take and plan accordingly.

So if statement "We estimated that project X will take 6 months" is meaningless then how should estimates be stated? Try to use ranges and rephrase statement to "We estimated that project X will take 5 to 7 months". Sometimes, when uncertainty is big, the range between best and worst case can be significant, but wide range is not a result of bad estimation, it's a result of lack of specification!

Data and definition of "good estimate" used in this post are from "Software Estimation: Demystifying the Black Art" book.

No comments: