Test Automation Patterns What is a pattern? Dorothy Graham

Test Automation Patterns
Prepared and presented by
Dorothy Graham
@DorothyGraham
[email protected]
www.DorothyGraham.co.uk
© Seretta Gamba, Dorothy Graham 2015
1
What is a pattern?
–  a generally reusable solution to a commonly
occurring problem
–  a description for how to solve a problem that can
be used in many different situations
–  a pattern is NOT
•  a finished solution that can be ‘plugged in’ or
transformed directly into code
•  prescriptive
•  a set of steps
–  a pattern needs to be implemented within context
2
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
Where did these automation patterns
come from?
–  Experiences book published 2012
–  Seretta Gamba contributed Ch 21
•  Automation through the back door,
by supporting manual testing
–  Seretta is a developer, used to
using patterns
–  when she read the other chapters, she thought:
“Patterns! Patterns!”
–  she wrote up test automation patterns as a book &
asked me to join her
–  Word didn’t work very well, so we made a wiki
3
Test Automation Patterns wiki
–  ISSUES – describe problems in automation
–  PATTERNS – proven ways to overcome problems
–  Mind Maps of issues, patterns
–  Diagnostics and General issues: ways to find the
most relevant issues and patterns
–  Experiences (we want more of these)
–  acknowledgements, references, some failure
patterns
TestAutomationPatterns.wikispaces.com
TestAutomationPatterns.org
4
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
About these patterns (and the wiki)
•  the wiki has a limited set of patterns
–  and a limited “depth” for each pattern
•  patterns use other patterns
–  it can get rather complex
•  we are focusing on system-level automation
rather than unit test automation
–  See Gerard Meszaros’ book “xUnit Test Patterns:
Refactoring test code” for unit test patterns
•  these are test automation patterns, not test
patterns
5
6
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
Test automation Issues: 4 examples
•  NO PREVIOUS TEST AUTOMATION (Mgt & Gen)
–  Your first time automating, or automation has failed and
you are starting again.
•  BRITTLE SCRIPTS (Design)
–  Automation scripts have to be reworked for any small
change in the Software Under Test (SUT)
•  STALLED AUTOMATION (Process)
–  Automation has been tried, but it never “got off the
ground” or has now fallen into disuse.
•  UNREALISTIC EXPECTATIONS (Mgt)
–  Management has unrealistic expectations about what
test automation can & can’t do.
7
Some other Issues
•  ERRATIC TEST (Execution)
–  the automated tests seem to pass or fail randomly
•  LITTER BUG (Design)
–  Tests leave spurious data in databases, history
etc. that slows performance or hinders further
tests
•  UNSATISFACTORY QUALITY OF TEST
AUTOMATON (General)
–  Test automation works to some extent, but isn't
giving desired benefits or is beset by problems
8
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
9
Test automation Patterns: 4 examples
•  SET CLEAR GOALS (Mgt)
–  Define the project objectives from the very beginning
•  DO A PILOT (Mgt)
–  Start a pilot project to explore how to best automate tests
on the application
•  TESTWARE ARCHITECTURE (Design)
–  Design the structure of your testware so that your
automators and testers can work as efficiently as possible
•  GOOD DEVELOPMENT PROCESS (Process)
–  Use good development practices for test code; the same as
developers [should] use for production code
10
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
Some other patterns
•  FAIL GRACEFULLY (Exec)
–  if a test fails it should restore the system and
environment so successive tests are not affected
•  KILL THE ZOMBIES (Exec)
–  regularly delete or archive test cases, scripts or test
data that are not used. Make sure that tests are
actually testing something
•  VERIFY-ACT-VERIFY* (Design)
–  The action to test is surrounded by two verifications
that check the initial and final state
•  thanks to Alexandra Schladebeck & team at BREDEX
11
Experiences from various people
•  AUTOMATION ROLES (Mgt)
–  Jen Leger – how they organised their team &
breakdown of activities
•  SET STANDARDS (Process)
–  Jochin Van Dorpe had 2 years’ worth of
automated tests with meaningless names, always
green – devised structure & naming conventions
•  TEST SELECTOR (Design)
–  Marcus Merrell share the tags they use to group
tests to choose which to run
12
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
More experiences from various people
•  OBJECT MAP (Exec)
–  Freddy Vega gives an example Object Repositiory
to map GUI elements to meaningful names
•  AUTOMATE GOOD TESTS (Design)
–  Bryan Bakker describes tests for reliability
•  We want YOU – to add your experiences to
the wiki please
–  to make it better for everyone!
13
Conclusion
•  we made a wiki about test automation patterns
–  we hope it will be useful for you
•  the wiki is free to view
–  to edit or add – request membership, we invite
anyone who asks!
•  please help make the wiki more useful
–  by adding your experiences
•  of issues or patterns
•  we welcome feedback about the wiki
14
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk
More information
•  TestAutomationPatterns.org
•  downloads www.DorothyGraham.co.uk
–  automation problems survey, articles and papers
–  list of test execution & framework tools
•  email [email protected]
–  feedback on the wiki
–  automation ROI calculator spreadsheet
•  blog http://dorothygraham.blogspot.com
–  including automation, DDP, certification
•  twitter
–  @DorothyGraham
15
[email protected]
[email protected]
© Seretta Gamba and Dorothy Graham 2015
www.DorothyGraham.co.uk