Why do we write test cases?

Replies to This Discussion

Permalink Reply by Pradeep Soundararajan on July 3, 2008 at 4:31pm

We - I don't.

Why I don't write test cases?

What matters to find bugs is ideas to test the product. I use the heuristic approach, guide word approach, oracle approach and thereby explore.

In India, am I able to survive without doing it?

Yes, I have been surviving for the past 5 and half years. I have been successful enough to the extent that I have the highest competitive advantage over most testers in India.

Why do people not like me write test cases?

They write because:

a) They ( those who advocate ) have been asked to.
b) They ( those who want to follow it ) want to write because that's the only way they know to test.
c) The process mandates them to do that.
d) They think that's how they can achieve good results in testing.
e) They don't want to know they explore.
f) They are worried about accountability and achieve accountability by sacrificing information.
g) and more

Why don't they try other ways?

Because:

a) They don't want to learn. ( although they might disagree )
b) They think it doesn't work in India. ( and don't try )
c) They know its hard to do good exploratory testing ( and they want easy stuff )
d) They don't want to know other ways.
e) They don't think/believe it exists.

Would this situation change?

It might change as more people like Sharath , Ajay, Sangeetha, Bhargavi ( all of Test Republic and more ) demonstrate to the world how better testing they currently do by exploring versus the approach they were asked to follow earlier.

Is there something more from Pradeep on test cases and why it's not a good approach?

http://testertested.blogspot.com/2007/11/how-to-show-product-qualit... and http://testertested.blogspot.com/2007/10/test-case-passed-who-cares...

Ok, let me know one reason that I can confidently argue with anyone that test cases hinder a tester?

Those who execute ( write, advocate ) test cases ( or scripts ) don't continue to test once they push themselves to managerial positions such as Leads and Managers because they are bored.

How different are exploratory testers?

Michael Bolton ( of Test Republic ) execute tests and finds tons of bugs having over 20 years of experience. James Bach is hired to test products post an experience of about 27 years. Ben Simo ( of Test Republic ) is another great example.

What are the important things of a test?

The precondition, steps, and NOT the expected result.

Why NOT the Expected Result?

Typically the expected results come from a specification or requirement document. No spec or req document are complete and hence testers get misguided or biased to that single point and miss out other bugs.

So, how do exploratory testers do it?

They run oracles in mind. An oracle is a principle or mechanism by which we humans identify problems.

And why is a PASS/FAIL thing missing?

It's missing because Rapid Testers ( www.satisfice.com/rst.pdf ) ask a question: Is there a problem here? and don't think of a test passing or failing. It's too narrow minded to think of a Pass/Fail.

A customer wouldn't be happy to know if all test passed but would like to know if most bugs were found.

Pradeep, just because you are an exploratory tester, are you professing its a great approach?

If a context suits to have test scripts then I would do that. Most contexts that I have been in ( similar to that of most testers in India ) do not demand me to write scripts/test cases.

What do exploratory testers have that scripted testers don't have

They have a lot of fun performing test execution!

What are the success stories of exploratory testers in India

You have a post in Test Republic blog from Ajay - 450 bugs in a short span of time and his company his amazed. Bhargavi, a tester who reported to me found most number of bugs for a quarter year when she joined TriVium and no experienced tester had found as many as she found. Pradeep also helped an organization reach a level that their customers couldn't report a bug for 8 month period he consulted them.

Ok, all this is great but it will not work for me

Those people you see above are not the ones who thought like that. If you think it will not work for you - it will not.

What if my manager asks me not to do it?

First, you dont need to let anyone know that you are exploring ( if they would object for it ) Your career is more important. The company you work for wouldn't cry or bother if you become an unsuccessful tester in the future and it is your duty to become successful. Second, for every scripted test you can run a quick exploratory test and find those bugs that your test case / spec wouldn't help you in catching it.

How can I learn to do exploratory testing?

There are a number of ways - www.satisfice.com/rst.pdf , www.testingeducation.com , www.developsense.com are a few that has helped me.

All this is good but I don't want to follow

You wasted time reading this. Go back and write test cases.

Sharat asked: A test case fails to list all pre-conditions, a test case fails to list all steps for execution,...

There are a number of things we humans or computers we use can control and things we can't control or we don't know to control. Those things that we can control and fall under "important" aspects of a test is what typically people want to document as pre-condition and steps BUT they go beyond it as they don't know what are the things they can control.

Would exploratory testers be able to reproduce bugs?

Yes, if they practice Session Based Test Management. Googling on that can help you further.


--
Pradeep Soundararajan - http://testertested.blogspot.com - pradeep.srajan@gmail.com

▶ Reply to This

Permalink Reply by sundara rao on July 4, 2008 at 2:10pm

may be test consultants dont write test cases but Tester working in a proces based companies will write test cases.

▶ Reply to This

Permalink Reply by Rajathi Rajesh on July 4, 2008 at 2:34pm

Any one doing formal testing, MUST have Test Cases!! Otherwise it becomes adhoc and Person dependent.

▶ Reply to This

Permalink Reply by Pradeep Soundararajan on July 4, 2008 at 3:33pm

Any one doing formal testing, MUST have Test Cases!! Otherwise it becomes adhoc and Person dependent.

What's wrong in testing becoming person dependent?

Most testers in this test republic forum would like to become expert testers. Expert testers is a person specific thing. If you are able to become a person dependent one in a project, you get more money. All testers in Test Republic want to make more money.

-- Pradeep

▶ Reply to This

Permalink Reply by Shrini Kulkarni on July 10, 2008 at 4:37am

How can something as intellectually demanding and critical thinking intensive work like testing be "person independent"? This craze of making things "person dependent" comes from those "process driven", factory model software (services) companies where managers keep preaching others saying "process should be standardized - no one is indispensible".

Rajathi - will your thinking process change if you wear hat of a tester ... It appears that you mostly think like a manager... Am I right?

I suggest kindly stop thinking on the lines of making testing "person independent" - it can never be - that too by a set of documented procedures called test cases.

BTW, do you know the meaning of word "Adhoc" ? Hint - adhoc does not mean sloppy and sub-standard.

Shrini

▶ Reply to This

Permalink Reply by Michael Bolton on July 10, 2008 at 10:08am

This craze of making things "person dependent" comes from those "process driven", factory model software (services) companies where managers keep preaching others saying "process should be standardized - no one is indispensible"

I think this way of thinking comes from the fear that if we don't document that we're going to test for a specific risk, then someone might forget to test for that specific risk, or that we won't be able to demonstrate that we tested for that risk. That is, the goal is to be able say "no matter who tests, we'll find this problem if it exists--and if it escapes, we'll be able to show that we acted responsibly."

Shrini, you're the person with whom I associate the idea of "goal displacement". I had heard the term before, but I heard it from you, it finally stuck in my head. So let's consider how the goals might have become displaced.

The goal is not to write out a step-by-step set of instructions to make sure that the tester finds problem X in a specific way; he goal is to make sure that the test is person-independent.

But wait a second; the goal isn't really to make the instructions person-independent; the goal is to make sure that a tester will find Problem X if it's there.

But still, hold on; the goal is not to make sure that a tester finds Problem X; the goal is to make sure that the product doesn't ship with Problem X in it.

Yet wait a second; the goal is not to make sure that the product doesn't ship with Problem X in it; the goal is to make sure that the product doesn't ship with Problem X or any problem as bad as Problem X, like Problem Y, or Problem Z or (and this is the kicker) some problem that we haven't thought of.

To accomplish that last goal--the italicized part--I think that heavyweight preparation reaches a limit, because it separates design, execution, and learning about the product and the problem space.

For this reason, I believe that we need to work on tester skill, and on providing testers with instructions that are sufficiently specific and sufficiently general to guide them find important problems quickly. So how do we do that? I suggest that we do it by tightening the feedback loop on test design, test execution, and learning. That means continuously attempting to reduce the need for documentation and planning to a bare minimum--not necessarily eliminating it, but reducing it to the absolute lower limit of what's needed to find problems quickly.

I'd like to hear from Rajathi on this suggestion.

---Michael B.

▶ Reply to This

Permalink Reply by Shrini Kulkarni on July 10, 2008 at 3:49pm

Michael - I could not have said any better.

I wrote 2 part post on this topic "Goal displacement"

http://shrinik.blogspot.com/2008/04/when-method-dictates-outcome-an...

http://shrinik.blogspot.com/2008/05/when-method-dictates-goal-see-g...


In a recent incident - I was to prepare a bid for an automation project. My boss told me that the proposal I prepared is not what he wanted as the proposal can be compared with other bid by the compitators (that was the requirement from client side). For that I responded - what client wants - a proposal for automation that solves their impending problem or a proposal for automation that they can use to compare bids accross the bidders. He had no response ... after a pause .. he said - "Both".

Can we do both in a single proposal? I could not do it ... I kept on my line of thinking that is focused on the "main" goal - automation that solves the problem. Finally, we won the bid as our approach of focussing on the goal was key.

So it is important for all testers to keep asking "what is my goal", "what problem am I trying to solve" and stay focussed on it. It is easier to get carried away from other people who have their own intentions and goals.

As tester we often serve multiple stakeholders - it is important to recognise who the stakeholder - what he/she values and how your service to him/her can help....

Rajathi -- where are you -- every one is waiting for your views


Shrini

▶ Reply to This

Permalink Reply by sharath on July 10, 2008 at 4:17pm

Congrats Shrini on winning the project.

That means continuously attempting to reduce the need for documentation and planning to a bare minimum--not necessarily eliminating it, but reducing it to the absolute lower limit of what's needed to find problems quickly>>

I was once moved to a project which was in acceptance stage, and asked to come up with test cases which could help developers test better/faster and customers understand the features tested at the offsite. The project previously did not have any testers. Another requirement was not to use much English, since customer’s first language was not English.

I used excel and created matrix like test cases, with almost minimum English words, which helped both the developers and customers. Client was very happy at the document format and the content it provided, that he requested to use it for similar scenarios.

- Sharath.B

▶ Reply to This

Permalink Reply by Michael Bolton on July 10, 2008 at 7:34pm

Matrices are certainly efficient and valuable for some kinds of tests, especially functional and domain tests.

One of the most effective test tools I ever had was one I developed in Excel. it was also the most challenging programming project that I ever developed from concept to implementation. I gave Excel's features a pretty good workout--heavy use of array formulas, multi-dimensional sheets, lookup tables, conditional formatting, working around some of Excel's bugs in logical operators in business rules... This proved a fantastic means, for me, of learning things about Excel and the business rules for the application in question. But because of the tight feedback loop between the application and the oracle, I learned two more important things: first, that is was indeed possible to model the application in Excel (which several of the project leaders had claimed wasn't possible); and that if I had to do a similar thing again, I've learned the skills to do it more rapidly.

The first set of test ideas that went into the model came from a matrix rather like what I think Sharath is describing above. The ideas themselves were loosely described, with some data explicitly specified. The nice thing about this particular project was that, by the time I had finished building the model, the data and the details of each transaction could be dynamic; that is, because I had programmed the business rules into the master worksheet, I could use that sheet as an oracle for any transaction that came to mind, when it came to mind. It was also highly useful in that when I wanted to capture a transaction or a scenario for future reference, all I had to do was snapshot and copy the sheet. Thus, even though the idea for the test might come to me in a moment, there was a very rapid means of showing and telling what that test idea was. It was a very powerful exploratory tool.

Meanwhile, other testers were executing scripted tests and finding problems. But those scripts rarely helped them to find a problem in the application. There were two kinds of problems that these testers found, in the main: 1) problems in the application that the scripts didn't hint about; and 2) problems with the scripts themselves. LOTS of problems with the scripts. However, these other testers, so far as I could tell, didn't learn much about the application, because (as usual) the scripts told them what to do, and not why what they were doing was important.

---Michael B.

▶ Reply to This

Permalink Reply by Pradeep Soundararajan on July 4, 2008 at 2:37pm

@Sundarara Rao,

may be test consultants dont write test cases but Tester working in a proces based companies will write test cases.

I would like to refer you to this post of mine: http://testertested.blogspot.com/2007/12/jony-jony-yes-papa-followi... in which I wrote what exploratory testing helped in achieving in a highly process oriented CMM Level 5 company I worked for.

Moreover, if you become a consultant would you suddenly start doing something that you were not doing earlier? You would want to do things that you did in the past and a test consultant when hired becomes a tester on the project.

-- Pradeep

▶ Reply to This

Permalink Reply by Jagan Mohan KVN on March 11, 2009 at 7:31pm

You are right, they talk more :-)

▶ Reply to This

Permalink Reply by Michael Bolton on July 3, 2008 at 10:39pm

One reason is to identify something important that we want to test for. Another is to help in developing a set of ideas as to what we might want to test. Another is to help make these ideas more transferrable between testers, or to make it easier to account for our work. Sometimes "write" is just another word for "invent", without recording. (Sometimes it's just another word for recording without inventing. That's a joke, but as with all jokes, there's some truth in it.)

But it's also useful to consider what we mean by "writing test cases". What does a test case look like? The answer you have for that will have a great bearing on the answer to your original question.

---Michael B.

▶ Reply to This

• ‹ Previous
• 1
• 2
• 3
• …
• 11
• Next ›
• Page