Replies to This Discussion

Permalink Reply by Pradeep Soundararajan on June 29, 2009 at 7:55am

Here is a story: I have had similar challenges to that of your explanation here. I was a rookie test manager then and I had to provide estimates to the Vice President who also was the Head of Development.

I think I initially tried helping him understand that any number, random or calculated ( using available magic formulas ) wouldn't be of help unless we perform those tasks over a cycle and estimate what time the next cycle might take. However, having used the magical formulas and making the magic work ( by compromising the quality ) he probably wasn't fine with my suggestions.

It then occured to me that I had read MB's blog post on Estimation - http://www.developsense.com/2007/01/test-project-estimation-rapid-w... whose idea I used to derive at estimates. I used to sit for half a day with my VP to provide estimates and then, I changed my style to half an hour.

It worked great wonders. We had more time testing together and more information to provide. When we provide blockers, the whole release plan changes. Somehow ( No, The Rapid Testing ) always used to be handy for me and my team to help provide valuable information faster that constantly helped in changing the plan to something more meaningful. After working on a couple of releases with the Vice President, I helped him understand, it wasn't worth spending so much time on estimation.

I think key stakeholders for estimation think an indicator for your estimation to be right is only when you have spent a lot of time and resources on it to arrive at numbers.

And those who execute the work think "if we go too beyond or twoo below the estimated time, we might have to spend more time answering questions in a meeting.We better do things to ensure we are close by to the estimated time"

These are some heuristics ( based on my experience) :

1. Give the number they are looking for to start a good relationship with them (whoever is asking you).
2. List your reasonable assumptions that influence the numbers you provided.
3. Work on two cycles. One that considers the number you quoted and the other that doesn't.
4. Provide them the difference of quality of work compared to both approaches or ask them to judge it.
5. Ask them, "We have more than one way of working towards the estimation and now that you have witnessed it, What do you prefer?"
6. Ask them, "Which is more practical and helpful to you?"
7. If they are insist their way to be followed (ignoring the better way), well you know whom you are working with and what works for them. That's valuable to you and me.

• ▶ Reply

Permalink Reply by Jaya on June 29, 2009 at 1:53pm

Test Estimation without knowing complete requirements is a challenge for most companies. The best approach is dependent on the project and the experience of the personnel involved.

I would suggest that first the project should be checked for its risk i.e. if it is banking, gaming, or some life critical software. A life critical project would definitely require extreme testing and needs quite good amount of testing time than a gaming project of bigger size.

Secondly, the estimation should be done keeping in mind the past experiences of the employees. If an employee has no past experience in testing a finance project, then he might require double the testing time than a person who has domain knowledge in the field.

Thirdly, the project outline/ any specifications given by customers should be divided into modules. These modules should further be broken down into smaller business cases and into a collection of small tasks for which estimates can be made with reasonable accuracy.

Also, we always keep some buffer time in our testing cycles. This buffer time will be used if a large number of bugs are found in the software. These large numbers of bugs require some development time and then some time will be needed for regression and retesting.

In the third step for making estimates, one can track the test efforts that worked well for similar projects in the company. The expected time of the new project’s modules would be based on the information available from other old projects like number of external interfaces, unit testing done by dev team etc.

In the end, this estimate is based on experience, and is not necessarily successful.

• ▶ Reply

Permalink Reply by Jay Sharma on August 2, 2011 at 5:33pm

Well said Jaya. It all depends on your level of experience but certain facts do take toll while estimating. Like no of screens/no of fields in each screen, no of roles, etc. As stated by about the modules exactly dividing req into separate modules gives clear picture. 

At the end your thought process is good. 

• ▶ Reply

Permalink Reply by Anna Borovcová on July 12, 2011 at 9:32pm

Here are some excelent points, but I think it doesn´t provide enough answers to the problem.

To sum what was already said here or in the linked post Pradeep provided:

• We do not decide when to stop testing - when the product is ready to be shiped. Someone more important do it, we will just provide information to them untill they think they have enough informations.
• We need to provide some estimation at the begining, but informed them it is not precise and do not try to fulfill the estimation.
• We should not spend to much time on the estimation, but we need to think about risks, experiences and divide into smaller tasks.

 

All of this is true, but there could be said much more about this topic such as:

• use heuristic,
• use development estimation,
• define assumptions for your estimations.

 

Use heuristic:

First you can ask people how much time it will take, but they never think of everything, even you will forgot about lot of tasks and problems. You can double or triple the guess, but it will be difficult to defend it.

To spend less time on estimation and have realistic number it is best to use heuristic and find similar project from past.

If there is not any, it can be usefull to find what the estimation is for development.

 

Use development estimation:

In development they do the similar thing as we. They design how the code should work i low lavel detail, we describe test cases and prepare data. They write code, we do test or write automation tests. They analyze code, we analyze results. In both cases there is management throuth similar amount of time.
So we should estimate it will take the same amount of time to be really confident about the product.
This is estimation is usefull, do not take more than an hour to make, precise, but has a
big limitations:
a) It works only for projects were the change has little influence to other systems (Change of one line could mean retest whole system)
b) It is for bigger critical projects. In non critical projects, the quality is not enough important to test so deep.
c) This estimation works for the whole testing process. If you want to estimate only business testing, when other department or company delivers it, this is no use for you.

 

Define assumptions for your estimations

But one thing I found the most important is work with assumptions. When you provide the estimation say on what assumptions you based it. How many bugs you expect, how many problems and with what, how many funtions do you expect, how many specification changes and so on.

In other terms if you say the estimation is valid only within these limits and nobody can blame you when the reality will be somewhere else, as several of these assupmsons will always be broken.

Because it really is just fortune-telling and you need a crystal ball or big luck to have your estimation precise.

(Or you can fix the estimation and have quality and scope of your testing as variable, but is really your estiomation worth doing bad job? )
 

• ▶ Reply