Exercise 1: Test case Hungama

Replies to This Discussion

Permalink Reply by M.V. Manoj on August 11, 2008 at 3:06pm

Think of a test case you executed in the past that helped you find a bug. Think of reasons why and when the same test case could not have helped you find a bug and put them as a list.>>

#1. Because the Defect which was found with that Test Case got FIxed. Later when I executed the same Test Case, it did not find a bug.

#2. Because of Technology change in the subsequent release (Our product moved from Thin Client to Thick Client), the Test Cases which found Defects in previous release became invalid for the current release.

#3. Because of Design change in the functionality, the Test cases which found Defects became invalid after the New Design implementation.

#4. There is also an instance where in I had logged a Defect against a requirement but on clarification and getting more information from Product Manager it turned out to be desired behaviour.

#5. Because the Test Case that found the bug was Automated and the automation script could not identify the defect, the Test Case did not find the bug.

▶ Reply to This

Permalink Reply by Pradeep Soundararajan on August 21, 2008 at 6:14pm

@Manoj,

#1. Because the Defect which was found with that Test Case got FIxed. Later when I executed the same Test Case, it did not find a bug.

I thought you found the bug and the test idea in the test case might have helped you found a bug. Did you find the bug or the test case?

▶ Reply to This

Permalink Reply by M.V. Manoj on August 22, 2008 at 10:06am

:-)

You are right, it was me who found the bug.

"The test doesn't find the bug. A human finds the bug, and the test plays a role in helping the human find it." -- Pradeep Soundararajan


-Manoj

▶ Reply to This

Permalink Reply by Pradeep Soundararajan on September 1, 2008 at 12:23pm

For those of you who did not participate in this, I am not sure why because if you are a part of Rapid Software Testing group, I thought you must be craving for testing exercises.

However, I spill my beans here:

Think of a test case you executed in the past that helped you find a bug. Think of reasons why and when the same test case could not have helped you find a bug and put them as a list.

The test case I can think of which helped me find a bug at my first job is: Establish connection between two bluetooth gadgets and then move out of range.

This test idea that was developed into a test case like:

Connect device A to device B
Connect device A to device C
Move out of the 10 m range with incremental of one metre for every 5 minutes.

While this was a test idea that I could have run with more dimensions than what was written, it restricted me to think that moving out means, moving out by walking and not throwing the device 10 m far away - which is a possible end user scenario.

However, I found a crash with this test idea:

Here are reasons why I might not have found the bug:

1. I did not recognize the bug.
2. I failed to focus on some portions of the display where a bug appeared and disappeared.
3. The bug I was trying to find was fixed and I didn't notice other bugs because I was keenly watching out for a specific bug.
4. I did not have skills to find the bug.
5. I did not understand some aspects of what the tester who wrote the test cases meant by that.
6. I did not execute the test.
7. I did not know how to execute the test.
8. I didnt not ask someone if I executed that test or something else.
9. When the bug occured someone interrupted me and I was defocusing from the place where the bug appeared and disappeared.
10. I thought that was not a bug.
11. I suspected it to be a feature.
12. I suspected it to be something that is not worth mentioning.
13. Inattentional blindness caused me not to find it.
14. I failed to recognize the behavior and pattern of the bug.
15. I winked and the bug appeared and disappeared when I winked.
16. I was asked to ignore such a behavior because I was misguided that it was appropriate.
17. I didn't have an oracle in mind that could have helped me find it.
18. The expected result in the test case was misleading.
19. The precondition documented in the test case was something that I could not have a control over.
20. I didn't adhere to follow the exact step by step procedure suggested by the author of the test case document.
21. I thought I followed the exact steps but I might have missed one of the step and thereby ended up executing some other test.
22. I ran the test in a different environment.
23. I ran the test on a different state of the products since I was not aware of the states.
24. I ran the test on a different build.
25. I ran the test on a device different from the one which helped in the finding the bug.

and more...

Now, doesn't that sound to be self revealing of so many things that could happen when we are executing tests. I created this exercise long back and tested it on a couple of testers.

This is an exercise of : How conscious we are about the things that might be misleading us when we are executing tests and how many ways can we think about our idea that a specific test that found a bug would help us find it again even if it was not fixed.

This also helps me understand that I can be fooled in so many different ways by the environment, bugs, people, words, test cases, my own understanding, assumptions, etc and makes me think I better question things that I hear, question things that I feel, question things that I see, question things that I want to see, question things that I want to hear, question things that I want to feel.

-- Pradeep

▶ Reply to This