AgileTestingOverview

1
An Overview of Agile
Testing
Tampere 2009
Lisa Crispin
With Material from Janet Gregory

Copyright 2009: Lisa Crispin
2
Introduction
•  Tester on agile teams since 2000
•  My teams:
•  Delight customers
•  Deliver production-ready value every 2
weeks
•  Drive development with tests, examples
•  100% regression test automation

3
Your Agile Experience
How about you?
  Managers, testers, developers,
ScrumMasters...?
  How long has your team used agile
practices/principles?

4
Takeaways
  Transitioning to agile testing
  How testing works on agile teams
  Agile testing tools and techniques
  Key success factors

5
Traditional vs. Agile SDLC
Phased or gated – eg. Waterfall
It 1 It 2 It 3
A B A B A B
C C
D
Time
Agile: iterative and incremental
- Each story is expanded, coded and tested
- Possible release after each iteration
Requirements
Specifications
Code
Testing
Release
5Copyright 2009 Lisa Crispin and
Janet Gregory

6
Testing as a Phase
  Code, then test
  Separate, siloed teams
  Heavyweight process
  Requirements, design documents
  Test plans
  “Traceability”
  Change review boards
  Testing “bottleneck” at end

7
Automation - After Coding
  Vendor tools
  Record/Playback
  Done by testers or automation team
  If developers automate unit tests – it’s after
coding
  Expensive

8
Exploratory Testing – “agile”
  Exploring reveals unintended consequences
and uncovers implicit expectations
  Elisabeth Hendrickson http://testobsessed.com/
2009/05/18/from-the-mailbox-fully-automated-gui-
testing/Careful observation
  Critical thinking, no requirements doc needed
  Diverse ideas
  Rich resources
  Tools, data, people
[source: Jon Hagar]

9
How Agile Changes Things
  Whole Team Approach
  Coding and testing are one process
  Feedback, collaboration key
  TDD/ATDD practices
  Test-infected developers, better tools, better-
designed tests

and Janet Gregory
10
Team Organization - Traditional vs. Agile
Agile: roles overlap Tester
ProgrammerDomain
Expert
Traditional: independent functional groups
Dev
BAs Tester

and Janet Gregory
11
Test Approach - The Agile Way
Project Initiation Get an understanding of the project
Participate in estimating stories Create Test PlanRelease Planning
Each Iteration
1
….
X
Write and execute story tests
Write and execute new functional test cases
Pair test with other testers, developers
Automate new functional test cases
Run automated regression test cases
Perform Load Test
Complete Regression Test
Perform UAT
Perform Mock Deploy
Participate in Release Readiness
System Test /
End Game
Participate in Release to Prod
Participate in Retrospectives
Release to Prod/
Support

12
The Whole Team Approach
  Team commitment to quality
  Team responsible for testing
  Testers get support, training
  Developers do testing activities
  Testers and developers pair
  Get the right people

13
Testing is Not a Phase
  Development = testing + coding
  Write test, write code, test
  At unit and acceptance test levels
  Automated tests
  Designed for efficiency,
maintainability
  Free time for exploratory testing
  Team plans testing activities
  All four “quadrants”

14
Agile Testing Quadrants

15
Feedback Guides Development
  Team uses feedback to improve
  Testers are expert feedback providers
  Feedback lets team make course
corrections

16
Feedback Examples
  Continuous Integration test results
  Faster is better
  Big Visible Charts
  Retrospectives
  Short iterations facilitate improvement
  Discoveries during exploratory testing
  Collaboration with users, customers

17
Core Practices for Success
  Continuous integration
  Frequent and fast feedback
  Coding and testing one process
  Test environments
  Manage technical debt
  Work incrementally
  Synergy

18
Where Is It Taking Us?
  Lines between roles blurred
  More focus on tests to drive coding
  Examples as requirements
  Find ways to control technical debt
  Tests pushed to lower levels
  Less end-to-end automation
  Automate based on ROI
  Maintainable tests
  Failures easier to debug

19
Test Automation Pyramid

20
Current Trends in Testing/Development
  Pomodoro technique for pairing, planning
  www.pomodorotechnique.com
  More exploratory testing
  As automated regression tests free time
  context-driven-testing.com
  Lean approaches
  Kanban, scrumban
  Poppendiecks’ books

21
More Current Trends
  Continuous Integration is a given
  More frequent releases
  Expanded test tool options
  Low-tech prototyping, tracking
  More early collaboration
  Behavior-Driven Development (BDD)

22
BDD/Narrative/Scenario Example
Given that the FitNesse wiki is a Wiki
Given that the FitNesse User is a Web User
Given that the FitNesse wiki was able to start for
the ﬁrst 7me
When the FitNesse User a9empts to go to the
Home Page
Then the content of the Home Page for the
FitNesse User should be Welcome to the
Wonderful World of FitNesse!
From: Antony Marcano, Andy Palmer,
testingreflections.com, pairwith.us

23
Scaling Up and out
  Larger organizations adopt agile
  My company has 28 Scrum teams
  More remote team members
  Need the best people, so…
  Hire them where they live
  Technology enables
  NOT remote test teams!

24
Test Tools
  Mix & Match open source frameworks/drivers
  Vendor tools getting lighter weight
  GUIdancer – object-oriented
  FitNesse/Slim/Fit widely used
  Framework for many other tools
  BDD tools growing in popularity
  Cucumber, easyb, nbehave
  GUI test drivers standardizing
  Watir, Selenium, Robot Framework

Testing Tool Spectrum (Paul King, asert.au)
Agile2009 - 25
©
A
S
E
R
T
2
0
0
6
-
2
0
0
9
Database
Drivers
DbUnit
DataSets
SqlUnit
groovy.sql
JPA
JDO
BigTable
JDBC
SOAP /
REST
Drivers
GroovyWS
XML-RPC
CXF
Axis2
JAX-WS
JAX-RS
Utilities
All Pairs Combinations
Polyglot languages
Logic programming
Threads, Parallel
concurrency libraries
Data-driven libraries
Networking libraries
XML Processing
Read/write files,
Excel / Word / CSV,
Reporting, Logging
Other
Drivers
FEST
Email
FTP
AntUnit
Telnet
SSH
Exec
Tools
iTest2, SoapUI, Twist,
GUIdancer, IDEs,
JMeter, Text editors,
Recorders, Build Tools,
CI
Web
Drivers
WebTest
WebDriver
JWebUnit
Tellurium
Selenium
HtmlUnit
Watij
HttpBuilder
Cyberneko
Runners
Native Groovy, JUnit, TestNG, Spock, EasyB,
JBehave, Cucumber, Robot Framework, Slim

26
Trends in Test Automation
  Record/Playback as “trainer” only
  Test architecture/design critical
  Planning and tracking tools – online and low-
tech
  Open source, agile tools explosion
  Plenty of good browser test drivers
  Other problems now being solved
  Scenario, Narrative style

27
Testing Techniques
  Team writes acceptance tests together
  Programmers and testers pair to automate
tests
  ROI guides automation
  Testers do exploratory testing
  Customers participate
  Pair with customers for ET
  Frequent demos

28
What Does It Mean for Us?
  Values more important than process,
practices
  Testers need to adopt new mindset
  Ramp up skills
  More creative thinking, problem-solving
  Do what works for your team
  Don’t blindly follow trends

29
Key Success Factors

30
Do Your Best Work
  Whole Team approach
  Testers, customers, developers collaborate
  Business examples, turned into tests
  Agile Testing Quadrants
  Test Automation Pyramid
  Exploratory testing

31
The Future is What You Make It
  Get out of your chair – see how you can help
  Learn a new test framework or scripting
language today - build your skills
  Lots of free resources, just takes time
  Leave your comfort zone
  Bring testing problems to the whole team
  Benefit from multiple skill sets
  How can your expertise help the team?

32
Questions?

33
Some Agile Testing Resources
 lisacrispin.com
 janetgregory.ca
 exampler.com
 agile-testing@yahoogroups.com
 testobsessed.com
 Antonymarcano.com

Copyright 2009: Lisa CrispinCopyright 2008 Janet Gregory, DragonFire
34
Bridging the Communication Gap
Specification By Example and Acceptance
Testing
Gojko Adzic

Copyright 2009: Lisa Crispin35
Now Available
Agile Testing: A Practical Guide for Testers and Agile
Teams
By Lisa Crispin and Janet Gregory
www.agiletester.ca

AgileTestingOverview

More Related Content

What's hot (20)

Similar to AgileTestingOverview (20)

AgileTestingOverview