Contents What is a testing technique? Black and White box testing

1
2
3
4
5
6
ISTQB / ISEB Foundation Exam Practice
Dynamic Testing Techniques
Contents
What is a testing technique?
Black and White box testing
Black box test techniques
White box test techniques
Error Guessing
White Box test design and
measurement techniques
• Techniques defined in BS 7925-2
– Statement testing
Also a measurement
technique?
– Branch / Decision testing
= Yes
– Data flow testing
= No
– Branch condition testing
– Branch condition combination testing
– Modified condition decision testing
– LCSAJ testing
• Also defines how to specify other techniques
Using structural coverage
Enough
tests?
Spec
Software
Tests
Results OK?
More tests
What's
covered?
Coverage OK?
Stronger structural
techniques (different
structural elements)
Increasing coverage
The test coverage trap
better testing
Function exercised,
insufficient structure
Functional
testedness
Structure exercised,
insufficient function
% Statement
% Decision
Structural testedness
100% coverage does
not mean 100% tested!
% Condition
Combination
Coverage is not
Thoroughness
Statement
Statement coverage
coverage
is normally measured
by a software tool.
• percentage of executable statements exercised by
a test suite
number of statements exercised
= total number of statements
• example:
– program has 100 statements
– tests exercise 87 statements
– statement coverage = 87%
Typical ad hoc testing achieves 60 - 75%
?
Example of statement coverage
1
read(a)
2
IF a > 6 THEN
3
b=a
4
ENDIF
5
print b
Statement
numbers
Test
case
Input
Expected
output
1
7
7
As all 5 statements are ‘covered’ by
this test case, we have achieved
100% statement coverage
Decision coverage
Decision coverage
is normally measured
by a software tool.
(Branch coverage)
• percentage of decision outcomes
exercised by a test suite
number of decisions outcomes exercised
= total number of decision outcomes
• example:
?
True
– program has 120 decision outcomes
– tests exercise 60 decision outcomes
– decision coverage = 50%
Typical ad hoc testing achieves 40 - 60%
False
Paths through code
12
12
123
?
?
?
1234
?
?
?
Paths through code with loops
1 2 3 4 5 6 7 8 ….
?
for as many times as it
is possible to go round
the loop (this can be
unlimited, i.e. infinite)
Example 1
Wait
for card to be inserted
be
inserted
IF card is a valid card THEN
display “Enter PIN number”
IF PIN is valid THEN
select transaction
ELSE (otherwise)
display “PIN invalid”
ELSE (otherwise)
reject card
END
Wait
Valid
card?
Yes
Display
“Enter..
No
Reject
card
Valid Yes Select
PIN?
trans...
No
Display
“PIN in..
End
Example 2
Read A
IF A > 0 THEN
IF A = 21 THEN
Print “Key”
ENDIF
ENDIF
3
– Cyclomatic complexity: _____
– Minimum tests to achieve:
• Statement coverage: ______1
• Branch coverage: _____3
Read
A>0
Yes
No
End
A=21
No
Yes
Print
Example 3
Read A
Read B
IF A > 0 THEN
IF B = 0 THEN
Print “No values”
ELSE
Print B
IF A > 21 THEN
Print A
ENDIF
ENDIF
ENDIF
Read
A>0
No
Yes
B=0
Yes
Print
No
Print
Yes
A>21
Print
No
End
4
– Cyclomatic complexity: _____
– Minimum tests to achieve:
2
• Statement coverage: ______
4
• Branch coverage: _____
Example 4
Read
A<0
Yes
Print
No
Print
Note: there
Read A
Read B
are 4 paths
IF A < 0 THEN
Yes
B<0
Print
Print “A negative”
No
ELSE
Print “A positive”
Print
ENDIF
IF B < 0 THEN
End
Print “B negative”
– Cyclomatic complexity: _____
ELSE
3
Print “B positive”
– Minimum tests to achieve:
ENDIF
• Statement coverage: ______
2
• Branch coverage: _____
2
Example 5
Read A
Read B
IF A < 0 THEN
Print “A negative”
ENDIF
IF B < 0 THEN
Print “B negative”
ENDIF
Read
A<0
Yes
Print
No
B<0
Yes
No
End
3
– Cyclomatic complexity: _____
– Minimum tests to achieve:
1
• Statement coverage: ______
2
• Branch coverage: _____
Print
Example 6
Read A
IF A < 0 THEN
Print “A negative”
ENDIF
IF A > 0 THEN
Print “A positive”
ENDIF
Read
A<0
Yes
Print
No
A>0
Yes
No
End
3
– Cyclomatic complexity: _____
– Minimum tests to achieve:
2
• Statement coverage: ______
2
• Branch coverage: _____
Print
1
2
3
4
5
6
ISTQB / ISEB Foundation Exam Practice
Dynamic Testing Techniques
Contents
What is a testing technique?
Black and White box testing
Black box test techniques
White box test techniques
Error Guessing
Non-systematic test techniques
•
•
•
•
Trial and error / Ad hoc
Error guessing / Experience-driven
User Testing
Unscripted Testing
A testing approach that is only
rigorous, thorough and systematic
is incomplete
Error-Guessing
• always worth including
• after systematic techniques have been used
• can find some faults that systematic techniques
can miss
• a ‘mopping up’ approach
• supplements systematic techniques
Not a good approach to start testing with
Error Guessing: deriving test cases
• Consider:
– past failures
– intuition
– experience
– brain storming
– “What is the craziest thing we can do?”
1
2
3
4
5
6
ISTQB / ISEB Foundation Exam Practice
Dynamic Testing Techniques
Summary: Key Points
Test techniques are ‘best practice’: help to find faults
Black Box techniques are based on behaviour
White Box techniques are based on structure
Error Guessing supplements systematic techniques