Replies to This Discussion

Permalink Reply by Alan Page on November 22, 2008 at 10:26am

For decision coverage, you need:
1 case where conditionA is true
1 case where conditionA is false
1 where conditionB is true
1 where conditionB is false

Why do you want to know? Are you asking for clarification of what decision and path coverage are, or just doing homework?

• ▶ Reply

Permalink Reply by Sajjadul Hakim on November 22, 2008 at 11:53am

So you really think you can actually quantify all the test cases required for "100%" path/decision coverage? Lets take a few scenarios.

How are you evaluating conditionA and conditionB? Can there be an out of bounds case for condition? When you are evaluating the expressions "c=c+1" or "d=d-1" or "e++", can there be data type max/min boundary errors? How are all these errors handled, since the execution path will probably break off to a different path not shown in your code snippet? Will there be concurrent executions of this code, since that would make the values even more unpredictable during your particular test? How about cases where you stop the application midway through the execution of all the statements? Would it matter if your computer goes into sleep mode in the middle of execution and returns? Does it matter what values you consider for the variables c, d, e, or the conditional parameters? Does these values come from system variables? Are there any domain implications of any of the values, for example if any of the values represent days in a month then certain days should be considered invalid? What does the "+" operator represent, i.e. is it an arithmetic addition, or a string concatenation or something else?

"100%"? I don't think we have even come close.

Regards,
Sajjadul Hakim
http://rapidtester.blogspot.com/

• ▶ Reply

Permalink Reply by Alan Page on November 22, 2008 at 6:57pm

You've identified the shortcomings of decision coverage and path coverage nicely. That doesn't change the definition of decision coverage (which was the original question). It's the difference between textbook testing and useful testing (which is why I asked why the OP was asking).

• ▶ Reply

Permalink Reply by Karan Raka on February 27, 2009 at 12:56pm

DFD is very useful to calculate no of testcases for statement and branch coverage .

• ▶ Reply

Permalink Reply by Michael Bolton on March 1, 2009 at 11:20pm

I presume that by "DFD" you mean "Data Flow Diagram".

I think the important distinction to be made here is the one between statement/branch/decision coverage and test coverage. Sajjadul questions whether you can have 100% line, or branch, or decision coverage. Well, you can. This mathematizes and simplifies the testing mission a great deal. However, as Sajjadul points out, this is just one model of coverage. In complex, open, interactive, multi-threaded systems, this kind of coverage is fine to have, as long as we remember to consider:

1) other known risks from inside our model, for which we might test;
2) unknown risks from inside or outside our model, for which we might explore;
3) timing;
4) state pollution and defenses against it, such as critical regions, semaphores, and the like;
5) the cost, especially the opportunity cost) of this kind of coverage, versus the value of this kind of coverage;
6) data types, including their extents and limits;
7) oracles, which might or might not reveal a problem (what happens if one of the values above overflows?)
8) operators, which might operate differently on different data types;
9) whether this is actual source code or pseudo-code, designed to model more detailed activities;
10) whether we have to focus on testing this code in isolation, or whether it will be covered by other tests;
11) whether A and B are really independent of one another (after all A's c might be the same as B's d or e, whether the programmer is aware of it or not);
12) whether the condition involved in A or B is affected by the variables inside the code blocks, or whether they're independent of those variables

...and a whole bunch of things that I haven't thought of.

Alfred North Whitehead said, "Seek simplicity, and upon finding it, learn to distrust it."

---Michael B.

• ▶ Reply

Permalink Reply by Meenakshi Manisha on March 4, 2009 at 5:22pm

I think its not DFD its control flow graph.

• ▶ Reply

Permalink Reply by Bj Rollison on March 2, 2009 at 12:40pm

Hi Shipra,

Getting back to your specific question about structural testing and specifically decision and path coverage. There is no standard calcuation for decision/branch coverage. Some people may suggest that decision coverage is 2 times the number of conditional clauses...but that is not correct. Decision coverage evaluates the true and false outcomes of each conditional clause in a simple predicate statement. So, to satisfy that requirement for this code you would simply need 2 tests in this example; one in which conditionA is true and conditionB is true, and a second test in which conditionA is false and condtionB is false. That would satisfy the requirements for decsion coverage.

McCabe proposed a theoretical formula to measure basis path coverage (not 100% path coverage) based on his cyclomatic complexity formula. v(G) = E - N + 2p. The hypothesis is that 100% path coverage is virutally impossible in any complex algorithm and the basis set is a set of linear independent paths and any other path through the algorithm is simply a superset of those linear independent paths. In this example, there would be 3 basis paths. However, because this is a simple code chunk, 100% path coverage in this example would require 4 tests as follows
1. conditionA == true and conditionB == true
2. conditionA == false and conditionB == false
3. conditionA == true and conditionB == false
4. conditionA == false and conditionB == true

Saijadul has some good points about behavioral testing, which is quite different as compared to structural testing. For example, whether or not the developer assigns the d variable to the c variable elsewhere in the code, or testing the boundary conditions of the variables c, d, and e has no import within your specified context of branch coverage testing for this example. (If variable c overflows and throws an unhandled exception then you simply don't evaluate conditionB which means you still need to test the true and false outcomes of condtionB for decision coverage.) Would additional behavioral and functional testing be important? Absolutely... and I suspect you already know that! I also suspect you understand the differences between control coverage, and test coverage.

Structural testing can be valuable when used by someone who understands the purpose and the limitations of this type of testing in the appropriate context.

BTW...I have a whole chapter on structural testing in our book How We Test Software At Microsoft .

• ▶ Reply

Permalink Reply by Shrini Kulkarni on March 2, 2009 at 12:57pm

A meta question : What does 100% coverage mean to you? most importantly what does that mean to your stakeholders - groups of people who will get impacted by success or failure of your testing service? How do you propose to use that metric? what information your are planning to extract out of it.

Alan covered this a bit in his reply Sajjadul and Michael exposed some subtle (and some not so subtle) fallibilities with the notion of "coverage" in general ...

What is your testing mission? Just to know about a formula for calculating path coverage/decision coverage? or to use it in a practical (real time scenario)

Can you confirm that this is a not a homework or an interview question .....

Shrini

• ▶ Reply

Permalink Reply by Bj Rollison on March 3, 2009 at 12:22pm

The question posed by Shipra was not an open-ended question. She explicitly asked about a specific type of structural control flow coverage; branch/decision coverage and path coverage. By asking a specific question she framed the context for the discussion.

Going 'meta' simply convoluted the request into a context-free open-ended discussion, and did not serve to answer her specific question. As I said the ideas of behavioral testing are good ideas, but did not speciifically address her question. Additionally, while the suggestions for additional behavioral and functoinal testing of the code add value in a complete testing scenario, none of them are relavant to the specific question of decison coverage or path coverage.

I cannot confirm this is homework or an interview question. I really don't care? Shipra asked a very specific question, and I responded with a direct answer that I hoped helped her. If this were an interview question and I asked Shipra how to design structural tests for decision coverage of this code during an interview and she responded with the meta hand-waving such as cost of this type of testing, sleep mode interrruptions, or oracle nonsense I would have deduced that while she can think tangentially, she could not constrain her response or stay focused on the specific task at hand. Point here being...when asked a specific question, you should provide a specific answer.

Your question "what does 100% coverage mean to you?" it means many things because it is a completely open-ended question because there are many types of coverage. Which specific type of coverage are you referring to?

• ▶ Reply

Permalink Reply by Shrini Kulkarni on March 3, 2009 at 1:11pm

Hi BJ,

>>>> The question posed by Shipra was not an open-ended question.
Shipra's question, in my opinion, has raised some related questions. It is in a generic form "How to measure blood pressure". Possibly there are two ways of looking at such class of questions. One - just respond to "how" part of the question and walk off. Another is probe "why, what, what else" about "how" of the question and seek to answer the question in a broader and holistic perspective. It appears you opted for first and myself, Michael, Sajjadul appear to have opted for the second. I really did not care (thought) about whether questions is open ended or not. I just sought to understand the background.

>>> By asking a specific question she framed the context for the discussion.
I do not think she provided the context to the discussion. she merely stated the question without any background. Taking the example of "blood pressure measurement" - Shipra, in my view, asked "tell me how to measure blood pressure of a human". This articulation, in my opinion, lacked context information.

>>> Going 'meta' simply convoluted the request into a context-free open-ended discussion, and did not serve to answer her specific question.

Only Shipra can tell whether any of our answers have served her *specific* question. All I can say is "going" meta, has only sought to add context to otherwise context free discussion. Does that made this discussion open ended? I am not sure.

>>>> As I said the ideas of behavioral testing are good ideas, but did not speciifically address her question.
Questions (meta) posed by me have nothing to with "behavioral" testing (or to structural testing). Shipra question had a problem statement and lacked the context. I sought to discover the context by asking meta questions. That is all.

>>> I cannot confirm this is homework or an interview question. I really don't care?
Unfortunately some us care. Few of us on this forum are "sensitive" to homework and interview questions. We attempt to treat such questions in a different manner and encourage the person asking homework/interview question to do his homework and the post here.
Pradeep? Sharat? Bhargavi? Michael B?

>>>I asked Shipra how to design structural tests for decision coverage of this code during an interview and she responded with the meta hand-waving such as cost of this type of testing, sleep mode interrruptions, or oracle nonsense I would have deduced that while she can think tangentially, she could not constrain her response or stay focused on the specific task at hand.

If I were the interviewer (God help the interviewee) and everything else remained same - I have given a "GO" to the candidate who would ask meta questions that I asked here and not before a follow-up question is with a more specific example to judge the candidate's ability to constrain the response with respect to specific job in hand. Because I value diversity in thinking and a holistic picture of why to something and why not any other thing and not undermining the skill of being precise when situation demanded so.

We appear to have two differing view point about how to judge an interview candidate. I am now sure, I would surely fail If I apply for Microsoft Redmond and you are the interviewer :) :)

>>> Point here being...when asked a specific question, you should provide a specific answer.
I think it helps in most of the cases, to probe the context - unless you are in a life and death situation. I am not sure that is the situation here. And yes, preciously for this reason, context to the question is important so that one can determine how quickly and how deep to answer.

>>>it means many things because it is a completely open-ended question because there are many types of coverage. Which specific type of coverage are you referring to?

100% coverage regardless of any type has a value to some one who seeks it. I am not specifically concerned about the type HERE. My question was "what does 100% of coverage of XXX type mean to you". I wanted shipra to answer that so that she understands the kind of information she is seeking.

In software testing, typically, we do not deal with perfectly deterministic problems like what is the sum of 2 and 3. Instead our problems are often open ended and require more following questions and lengthy answers. Our problems do not, typically follow "request-response" model but a intimate conversation model among a group of people.

Shrini

• ▶ Reply

Permalink Reply by Michael Bolton on March 3, 2009 at 9:17pm

By asking a specific question she framed the context for the discussion.

Did she? Or did she ask a sufficiently context-free question that it seeded a discussion?

Point here being...when asked a specific question, you should provide a specific answer.

By that reckoning, I perceive Alan did that, so there should have been no reason to recapitulate his answer or to amplify on it.

\We all (Sajjadul, me, Rob, Shrini, and you) attempted to add more context to the question and to our answers. We're all trying to help, right? What else is a discussion forum for?

---Michael B.

• ▶ Reply

Permalink Reply by shahjerome ambrose on March 3, 2009 at 9:57am

Path coverage can get complecated especially if the code extend to the thousands and you need to connect Unit and Integration test. Draw your code using binary branch and use the rules below.

In path coverage.....3 areas of product behavior will count

1. Positive outcome

2. Negative outcome

3. Exceptions or error capture

I like BJ Rollison's comment below and can be used as a good guide with the simple rule I mentioned above.

• ▶ Reply

• ‹ Previous
• 1
• 2
• Next ›
• Page