SlideShare a Scribd company logo

Build the Right Product Right: Transitioning Test from Critiquing to Defining

T
TechWell

Gerard Meszaros, an independent software development consultant, discusses shifting from critiquing to defining tests in agile development during his keynote presentation. He emphasizes the importance of automated testing, its benefits for documentation, regression, and exploratory testing, while also identifying challenges such as complexity and lack of design for testability. The presentation outlines how systems thinking, example-driven development, and proper retrospectives can improve product quality and team collaboration.

Software
Related topics:
Software Testing Insights
1 of 39
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
! ! K3# Keynote! 6/25/2015! ! 8:30!AM! ! ! ! ! “Build#the#Right#Product#Right:#Transitioning#Test# from#Critiquing#to#Defining”## Presented#by:# Gerard#Meszaros# Independent#Consultant# # # # # # Brought#to#you#by:# # # # # # # 340!Corporate!Way,!Suite!300,!Orange!Park,!FL!32073! 888D268D8770!E!904D278D0524!E!sqeinfo@sqe.com!E!www.sqe.com! !
! ! ! ! !!!!! ! !!!! An independent software development consultant and trainer, Gerard Meszaros has more than thirty years of experience in software and more than a decade of experience in agile methods. Gerard is an expert in test automation patterns, refactoring of software and tests, and design for testability. He has applied automated unit and acceptance testing on projects ranging from full-on eXtreme Programming to traditional waterfall development in a wide range of industries. Gerard is an expert in the implementation and customization of agile methods such as Scrum, lean, and eXtreme Programming. His book xUnit Test Patterns—Refactoring Test Code won a Jolt Productivity Award in the Best Technical Book category. Gerard Meszaros Independent Consultant
1 1 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Build the Right Product Right Transitioning Test from Critiquing to Defining Gerard Meszaros Independent Consultant CTO of FeedXL.Com StarCanada2015@gerardm.com These slides: http://StarCanada2015.gerardm.com 2 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Product & I.T. I.T. Embedded Telecom My Background Gerard Meszaros StarCanada2015@gerardm.com • Software developer • Development manager • Project Manager • Software architect • OOA/OOD Mentor • Requirements (Use Case) Mentor • XP/TDD Mentor • Agile PM Mentor • Test Automation Consultant & Trainer • Lean/Agile Coach/Consultant
2 3 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Product & I.T. I.T. Embedded Telecom My Background • Software developer • Development manager • Project Manager • Software architect • OOA/OOD Mentor • Requirements (Use Case) Mentor • XP/TDD Mentor • Agile PM Mentor • Test Automation Consultant & Trainer • Lean/Agile Coach/Consultant Gerard Meszaros StarCanada2015@gerardm.com 4 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Agile Dev I’m From Development and I’m Here to Help
3 5 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Johnny’s Report Card http://fabulesslyfrugal.com/report-card-freebies-for-good-grades/ 6 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com QA as Quality Assessment
4 7 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Dev Test & Fix “Battleship” Development Quality Assessment Bug Reports Software Requirements B6! Missed! Requirements Test 8 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Requirements Development Testing
5 9 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Testing Development Requirements Testing Development Requirements 10 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Testing Development Requirements Testing Development Requirements Testing Development Requirements Testing Development Requirements
6 11 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Automated Testing is Essential http://www.cordistech.com/ 12 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com http://www.etfile.com/blog/to-automate-or-not-to-automate-that-is-the-question/
7 13 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com empowering http://www.slideshare.net/b2bSEO/robots-replacing-humans-automation 14 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Value of Automated Tests Tests as Documentation Faster regression testing Faster feedback on impact of changes Better coverage of data combinations Test with data without seeing that confidential data Provides more time for exploratory testing Can be done off hours; reduces delay/resources Writing tests helps us understand code better Know when problem was introduced Fast sanity checking Performance or load testing Setting up complex scenarios for manual/exploratory testing Unit/module tests improve modularity design and API’s Help Business understand what we/they are asking for Courtesy of participants of Tuesday’s tutorial
8 15 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Automated Tests are Just Code That Tests Other Code Who Tests This Code? Who Should Write This Code? 16 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey https://lmsa.site-ym.com/news/221726/Sex-and-Gender-Medical-Education-National-Student-Survey.htm
9 17 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey (Part 2) 18 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey (Part 3) http://www.mcas.k12.in.us/Page/4826
10 19 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Insanity: •  Doing the same thing over and over again and expecting different results 20 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Project Retrospectives
11 21 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Prime Directive Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand 22 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Agile Retrospectives / Sprint Reviews
12 23 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Ineffective Retrospectives Stop Start More Actions 1. Do something 2. Do another thing 3. Do yet another thing 4. Do more stuff 5. Do yet more stuff 6. 24 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Ineffective Retrospectives Stop Start More Actions 1. Do something 2. Do another thing 3. Do yet another thing 4. Do more stuff 5. Do yet more stuff 6. How Do We Improve Retrospectives?
13 25 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com One Experiment per Iteration Stop Start More 1.  Hypothesis 2. Experiment 3. Analyse 4. Communicate 26 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Toyota Way
14 27 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s www.123rf.com 28 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Tests need to interact via the UI Test Automation is Hard! Product isn’t designed for testability Why? Why? Why? Tests are very complex
15 29 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? 30 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? QA is supposed to be “independent” Why?
16 31 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - Notation: + Increasing - Reducing 32 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - + + Notation: + Increasing - Reducing Reinforcing Cycle Balancing Cycle
17 33 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - + + Notation: + Increasing - Reducing Vicious or Virtuous Cycle 34 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? QA is supposed to be “independent” Why?
18 35 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test Automation – Systems Thinking Diagram Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability + + + + + + + + Tests are very complex + + QA is supposed to be “independent” + A Vicious Cycle 36 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Conway’s Law The Architecture of System will resemble the Organization that Built it. Development Team Test Team Business Team
19 37 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Resulting Architecture Specifi- cations Product Code Test Code Bus Test Dev 38 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Development flow Potentially Executable Specifications Example Interpreter Code Product Code Dev Test Dev DevTestBus Test
20 39 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Runtime View: Product Code Example Interpreter Code Report Card Potentially Executable Specifications Dev Test Dev DevTestBus Test 40 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Feedback Loops: Potentially Executable Specifications Product Code Example Interpreter Code Report Card What they say How they say it
21 41 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Feedback Loops: Potentially Executable Specifications Product Code Example Interpreter Code Report Card What they say How they say it Isn’t This “Just” Test Automation By Another Name? 42 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Tests vs. Checks vs. Examples?ref: http://www.satisfice.com/blog/archives/856
22 43 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Preparing Good Examples http://thepracticalleader.com/ 44 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example: Notifying of Bank Transactions Configure Notification Rules Suspend Notification Resume Notification Account Holder Process Transaction Transaction Settlement NotifybasedonchargeType NotifybasedonchargeContinent NotifybasedonchargeCountry NotifybasedonchargeCity&State SuspendNotification
23 45 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications •  Open MegaBank app •  Log in as “BobMa” with password ****** •  Click on “Manage Notifications” tab •  Click on “Add New Rule” button •  Select account “10035692877” •  Type “Default Rule” into field “rule name” •  Type “1000” into field “threshold amount” •  Click on “all transaction types” radio button •  Click on “all locations” radio button •  Click on “save changes” button •  … •  … 46 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications – 1/2 Given: User and Accounts Another attempt: When: Notification Rule is Configured Then: Notification Rule Configured and Active Then: Notification Rule is Active
24 47 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications – 2/2 Then: Expected Notifications When: The Transactions to be processed Another attempt: 48 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test Automation Pyramid Pyramid originally proposed by Mike Cohn Exploratory Tests Unit Tests Component Tests (API) System Tests (GUI)
25 49 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Unit Tests Component Tests (API) System Tests (GUI) Broad Narrow Detail HighLow Scope 50 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Broad Narrow Detail HighLow Scope Component Tests (API) System Tests (GUI)
26 51 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Component Tests (API) System Tests (GUI) Rules & Algorithms Multi- Use Case Workflows Incomplete Spec Too much detail Unmaintainable Detail HighLow Broad Narrow Scope Transactions (Use Cases) 52 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Too much detail Unmaintainable Changing Level of Abstraction/Detail Rules & Algorithms Multi- Use Case Workflows Transactions (Use Cases) Reduce Scope a Lot! ReduceDetailaLot Reduce Both a bit Incomplete Spec Broad Narrow Detail HighLow Scope
27 53 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example Given: User and Accounts When: Notification Rule is Configured Then: Notification Rule Configured and Active Then: Notification Rule is Active 54 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example “If it isn’t essential to conveying the essence of the behavior, it is essential to not include it.”
28 55 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Then: Expected Notifications When: The Transactions to be processed Refactoring Workflow Example 56 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example
29 57 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example 58 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow ExampleGiven: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications
30 59 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Given: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications credit credit 9991113333 Refactoring Workflow Example 60 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 9991113333 Refactoring Workflow Example Broad Scope (Multi-Actor); Minimum Detail (per Actor/Transaction); Given: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications credit credit
31 61 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Filling in the Details Rules & Algorithms Multi- Use Case Workflows Transactions (Use Cases) Incomplete Spec ReduceDetailaLot Reduce Both a bit Too much detail Unmaintainable Reduce Scope a Lot! Broad Narrow Detail HighLow Scope 62 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Business Rule Example Process Transaction Threshold per Charge Type Configuration
32 63 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Process Transaction Threshold per Charge Type Narrow Scope (Single Rule) High Detail (Everything that matters) Example: Configuration Given these rules Then: The answer should be Business Rule Example When we ask NotificationRequired? with this transaction: 64 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Life Cycle of an Example User Goal Feature Story Title Story Narrative Story Scenarios Story Examples Executable Examples Satisfied Examples Formalize Automate Develop Product Make Concrete Add Data Define Acceptance Criteria Work Breakdown Product Design & Validation Elaborate
33 65 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Who Does Activities on an Example User Goal Feature Story Title Story Narrative Story Scenarios Story Examples Executable Examples Satisfied Examples Formalize Automate Develop Product Make Concrete Add Data Define Acceptance Criteria Work Breakdown Product Design & Validation Elaborate DevTest Bus DevTest Bus Dev Test Bus 66 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test-After Architecture System Under Test Should we Notify?Process Transaction Configure Notification Threshold Notification Rules Transaction Interface Configuration User Interface Notification Log Workflow Test Do Notification. Bus Test Dev
34 67 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Architecture System Under Test Should we Notify? Do Notification. Process Transaction Configure Notification Threshold Notification Rules Notification Log Transaction Interface Configuration User Interface Workflow Example Workflow Keyword Interpreter Dev Dev Bus Test 68 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Transaction Interface Configuration User Interface Configure Notification Threshold System Under Test EDA – Business Rules Do Notification. Process Transaction Notification Rules Notification Log Notification Method Example Notification Rules Fake Notification Rule Example Should we Notify? Example Interpreter Example Interpreter Dev Bus Test Dev
35 69 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com EDA – Business Rules Should we Notify? Notification Rules Fake Notification Rule Example Example Interpreter Dev Bus Test Dev 70 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com After State – Back to Our Systems Diagram Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability + + + + + + + + Tests are very complex + + QA is supposed to be “independent” + X X X X X XXX
36 71 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com After State – Back to Our Systems Diagram Dev knows how (and cares) Examples don’t need to interact via the UI Automated checking is Easy! Automation is Dev’s problem Examples are automated before Dev starts QA is involved in product definition QA has time for Examples and Exploratory testing Product is designed for testability + + + + + + + + Examples are simple + + A Virtuous Cycle 72 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com My Challenge to You Stop Start More 1.  Hypothesis 2. Experiment 3. Analyse 4. Communicate
37 73 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 74 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Thank You! Gerard Meszaros StarCanada2015@gerardm.com http://www.xunitpatterns.com Slides: http://StarCanada2015.gerardm.com Call me when you: •  Want to transition to Agile or Lean •  Want to do Agile or Lean better •  Want to teach developers how to test •  Need help with test automation strategy •  Want to improve your test automation Jolt Productivity Award winner - Technical Books Available on MSDN:

More Related Content

PDF
System-Level Test Automation: Ensuring a Good Start
TechWell
 
PDF
Test Design for Fully Automated Build Architecture
TechWell
 
PDF
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
PDF
Testing Transformation: The Art and Science for Success
TechWell
 
PDF
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
PDF
Communication and Testing: Why You Have Been Wrong All Along!
TechWell
 
PDF
Use Layered Model-Based Requirements to Achieve Continuous Testing
TechWell
 
PDF
Transforming Your QA and Test Team
TechWell
 
System-Level Test Automation: Ensuring a Good Start
TechWell
 
Test Design for Fully Automated Build Architecture
TechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
Testing Transformation: The Art and Science for Success
TechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
Communication and Testing: Why You Have Been Wrong All Along!
TechWell
 
Use Layered Model-Based Requirements to Achieve Continuous Testing
TechWell
 
Transforming Your QA and Test Team
TechWell
 

What's hot (20)

PPTX
The Future of Testing
Perfecto by Perforce
 
PPTX
Exploratory Testing with JIRA | QASymphony Webinar
QASymphony
 
PPTX
Let's focus more on Quality and less on Testing by Joel Montvelisky
QA or the Highway
 
PPTX
Smarter Automation with Machine Learning & AI-Based Reporting
Perfecto by Perforce
 
PPTX
Continuous Testing: The Path Forward
Perfecto by Perforce
 
PDF
Test-Driven Development for Embedded C -- OOP Conference 2015, Munich
James Grenning
 
PDF
Cloudbrew 2019 observability driven development
Geert van der Cruijsen
 
PDF
Observability driven development
Geert van der Cruijsen
 
PDF
Test Driven Development for Embedded C
James Grenning
 
PDF
The art and joy of testing in production
Geert van der Cruijsen
 
PDF
Patterns of Automation: Simplify Your Test Code
TechWell
 
PDF
Try: Fail, Try: Succeed by Tim Grant
QA or the Highway
 
PDF
Acceptance Testing for Continuous Delivery by Dave Farley at #AgileIndia2019
Agile India
 
PPTX
Advanced Codeless Testing for Web Apps
Perfecto by Perforce
 
PDF
Blunders in Test Automation
TechWell
 
PDF
Testing in the Dark
TechWell
 
PDF
CRMUG UK November 2015 - Testing testing! Panel Discussion with Scott Durow, ...
Wesleyan
 
PDF
Solid c-accu2014.key
James Grenning
 
PDF
The Tester’s Role in Agile Planning
TechWell
 
PDF
DevOps the Big Picture for Testers by Joseph Ours
QA or the Highway
 
The Future of Testing
Perfecto by Perforce
 
Exploratory Testing with JIRA | QASymphony Webinar
QASymphony
 
Let's focus more on Quality and less on Testing by Joel Montvelisky
QA or the Highway
 
Smarter Automation with Machine Learning & AI-Based Reporting
Perfecto by Perforce
 
Continuous Testing: The Path Forward
Perfecto by Perforce
 
Test-Driven Development for Embedded C -- OOP Conference 2015, Munich
James Grenning
 
Cloudbrew 2019 observability driven development
Geert van der Cruijsen
 
Observability driven development
Geert van der Cruijsen
 
Test Driven Development for Embedded C
James Grenning
 
The art and joy of testing in production
Geert van der Cruijsen
 
Patterns of Automation: Simplify Your Test Code
TechWell
 
Try: Fail, Try: Succeed by Tim Grant
QA or the Highway
 
Acceptance Testing for Continuous Delivery by Dave Farley at #AgileIndia2019
Agile India
 
Advanced Codeless Testing for Web Apps
Perfecto by Perforce
 
Blunders in Test Automation
TechWell
 
Testing in the Dark
TechWell
 
CRMUG UK November 2015 - Testing testing! Panel Discussion with Scott Durow, ...
Wesleyan
 
Solid c-accu2014.key
James Grenning
 
The Tester’s Role in Agile Planning
TechWell
 
DevOps the Big Picture for Testers by Joseph Ours
QA or the Highway
 
Ad

Viewers also liked (17)

PDF
Large-Scale Agile Test Automation Strategies in Practice
TechWell
 
PDF
Agile Hacks: Creative Solutions for Common Agile Issues
TechWell
 
PDF
Implementing Agile in an FDA Regulated Environment
TechWell
 
PDF
Prevent Test Automation Shelfware: A Selenium-WebDriver Case Study
TechWell
 
PDF
Power Your Teams with Git
TechWell
 
PDF
Privacy and Data Security: Minimizing Reputational and Legal Risks
TechWell
 
PDF
Create Brainstorming Commandos for Creative Problem Solving
TechWell
 
PDF
The Issues Agile Exposes and What To Do about Them
TechWell
 
PDF
Managing Technical Debt
TechWell
 
PDF
Using DevOps to Drive the Agile ALM
TechWell
 
PDF
The Adventures of a First-Time Test Lead: An Unexpected Journey
TechWell
 
PDF
Now That We're Agile, What's a Manager to Do?
TechWell
 
PDF
Managing a Software Engineering Team
TechWell
 
PDF
Use Business Analysts for User Interface Design
TechWell
 
PDF
Testing in a Super-Agile Software Development Environment
TechWell
 
PDF
Developing a Rugged DevOps Approach to Cloud Security
TechWell
 
PDF
From Unclear and Unrealistic Requirements to Achievable User Stories
TechWell
 
Large-Scale Agile Test Automation Strategies in Practice
TechWell
 
Agile Hacks: Creative Solutions for Common Agile Issues
TechWell
 
Implementing Agile in an FDA Regulated Environment
TechWell
 
Prevent Test Automation Shelfware: A Selenium-WebDriver Case Study
TechWell
 
Power Your Teams with Git
TechWell
 
Privacy and Data Security: Minimizing Reputational and Legal Risks
TechWell
 
Create Brainstorming Commandos for Creative Problem Solving
TechWell
 
The Issues Agile Exposes and What To Do about Them
TechWell
 
Managing Technical Debt
TechWell
 
Using DevOps to Drive the Agile ALM
TechWell
 
The Adventures of a First-Time Test Lead: An Unexpected Journey
TechWell
 
Now That We're Agile, What's a Manager to Do?
TechWell
 
Managing a Software Engineering Team
TechWell
 
Use Business Analysts for User Interface Design
TechWell
 
Testing in a Super-Agile Software Development Environment
TechWell
 
Developing a Rugged DevOps Approach to Cloud Security
TechWell
 
From Unclear and Unrealistic Requirements to Achievable User Stories
TechWell
 
Ad

Similar to Build the Right Product Right: Transitioning Test from Critiquing to Defining (20)

PPTX
Tackling Strategic Engineering Challenges
Applause
 
PDF
Test Automation Strategies and Frameworks: What Should Your Team Do?
TechWell
 
PDF
CookpadTechConf2018-(Mobile)TestAutomation
Kazuaki Matsuo
 
PPT
SDT Blended Model V3 2009
Ed Kit
 
PDF
Creating a successful continuous testing environment by Eran Kinsbruner
QA or the Highway
 
PDF
Are Your Continuous Tests Too Fragile for Agile?
Parasoft
 
PDF
testingexperience19_09_12
Damian Buonamico
 
PDF
How to build confidence in your release cycle
DiUS
 
PPTX
Test Smarter Not Harder - how to design tests for continuous delivery
Melissa Benua
 
PPT
Quality Assurance & Testing in a glimpse
Tahmid Munaz
 
PPT
Universal test solutions customer testimonial 10192013-v2.2
Universal Technology Solutions
 
PDF
Manage Testing by Dependencies—Not Activities
TechWell
 
PDF
Automated Testing: QA Horizons
Prottay Karim
 
PDF
Software Testing Interview Questions & Answers | Edureka
Edureka!
 
PPT
TestIT Software Assurance
testitcouk
 
PDF
Top 50 Software Testing Interview Questions & Answers | Edureka
Edureka!
 
PPTX
Deploying DriveWorks Throughout the Organization
Razorleaf Corporation
 
PDF
Continuous delivery is more than dev ops
Agile Montréal
 
PDF
Stamp Out Agile and DevOps Bottlenecks
TechWell
 
PDF
Why Automation Fails—in Theory and Practice
TechWell
 
Tackling Strategic Engineering Challenges
Applause
 
Test Automation Strategies and Frameworks: What Should Your Team Do?
TechWell
 
CookpadTechConf2018-(Mobile)TestAutomation
Kazuaki Matsuo
 
SDT Blended Model V3 2009
Ed Kit
 
Creating a successful continuous testing environment by Eran Kinsbruner
QA or the Highway
 
Are Your Continuous Tests Too Fragile for Agile?
Parasoft
 
testingexperience19_09_12
Damian Buonamico
 
How to build confidence in your release cycle
DiUS
 
Test Smarter Not Harder - how to design tests for continuous delivery
Melissa Benua
 
Quality Assurance & Testing in a glimpse
Tahmid Munaz
 
Universal test solutions customer testimonial 10192013-v2.2
Universal Technology Solutions
 
Manage Testing by Dependencies—Not Activities
TechWell
 
Automated Testing: QA Horizons
Prottay Karim
 
Software Testing Interview Questions & Answers | Edureka
Edureka!
 
TestIT Software Assurance
testitcouk
 
Top 50 Software Testing Interview Questions & Answers | Edureka
Edureka!
 
Deploying DriveWorks Throughout the Organization
Razorleaf Corporation
 
Continuous delivery is more than dev ops
Agile Montréal
 
Stamp Out Agile and DevOps Bottlenecks
TechWell
 
Why Automation Fails—in Theory and Practice
TechWell
 

More from TechWell (20)

PDF
Failing and Recovering
TechWell
 
PDF
Build Your Mobile App Quality and Test Strategy
TechWell
 
PDF
Implement BDD with Cucumber and SpecFlow
TechWell
 
PDF
Ma 15
TechWell
 
PDF
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
PDF
Transform Test Organizations for the New World of DevOps
TechWell
 
PDF
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
PDF
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
PDF
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
PDF
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
PDF
A Business-First Approach to DevOps Implementation
TechWell
 
PDF
Databases in a Continuous Integration/Delivery Process
TechWell
 
PDF
Mobile Testing: What—and What Not—to Automate
TechWell
 
PDF
Cultural Intelligence: A Key Skill for Success
TechWell
 
PDF
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 
PDF
Scale: The Most Hyped Term in Agile Development Today
TechWell
 
PDF
Measure DevOps for Objective Continuous Improvement Practices
TechWell
 
PDF
Microservices and Docker at Scale: The PB&J of Modern Systems
TechWell
 
PDF
Automation Anti-Patterns: Deal with Them
TechWell
 
PDF
Put Agile to the Test: A Case Study for Test Agility on a Large IT Project
TechWell
 
Failing and Recovering
TechWell
 
Build Your Mobile App Quality and Test Strategy
TechWell
 
Implement BDD with Cucumber and SpecFlow
TechWell
 
Ma 15
TechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
Transform Test Organizations for the New World of DevOps
TechWell
 
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
A Business-First Approach to DevOps Implementation
TechWell
 
Databases in a Continuous Integration/Delivery Process
TechWell
 
Mobile Testing: What—and What Not—to Automate
TechWell
 
Cultural Intelligence: A Key Skill for Success
TechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 
Scale: The Most Hyped Term in Agile Development Today
TechWell
 
Measure DevOps for Objective Continuous Improvement Practices
TechWell
 
Microservices and Docker at Scale: The PB&J of Modern Systems
TechWell
 
Automation Anti-Patterns: Deal with Them
TechWell
 
Put Agile to the Test: A Case Study for Test Agility on a Large IT Project
TechWell
 

Recently uploaded (20)

PDF
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
PPTX
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
PPTX
history of c programming in notes for students .pptx
Vijayakumari829030
 
PDF
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
PPTX
Computer Software and OS of computer science of grade 11.pptx
GauravDahal9
 
PDF
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
PDF
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
PDF
System and Network Administration Chapter 2
jaramharo
 
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
PDF
Digital Systems & Binary Numbers (comprehensive )
fyfhqfhtv2
 
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
PPT
Introduction Database Management System for Course Database
ReinertYosua
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PPTX
Reimagine Home Health with the Power of Agentic AI​
AutomationEdge Technologies
 
PDF
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
PPTX
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 
PPTX
Transform Your Business with a Software ERP System
Canada Software Company
 
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
history of c programming in notes for students .pptx
Vijayakumari829030
 
wealthsignaloriginal-com-DS-text-... (1).pdf
skmiani786
 
Computer Software and OS of computer science of grade 11.pptx
GauravDahal9
 
T3DD25 TYPO3 Content Blocks - Deep Dive by André Kraus
Andre Kraus
 
Internet Downloader Manager (IDM) Crack 6.42 Build 42 Updates Latest 2025
itskinga12
 
2025 Textile ERP Trends: SAP, Odoo & Oracle
Aetos Technologies
 
System and Network Administration Chapter 2
jaramharo
 
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
Digital Systems & Binary Numbers (comprehensive )
fyfhqfhtv2
 
Upgrade and Innovation Strategies for SAP ERP Customers
Virendra Rai, PMP
 
Introduction Database Management System for Course Database
ReinertYosua
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
Reimagine Home Health with the Power of Agentic AI​
AutomationEdge Technologies
 
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
Embracing Complexity in Serverless! GOTO Serverless Bengaluru
SheenBrisals
 
Transform Your Business with a Software ERP System
Canada Software Company
 

Build the Right Product Right: Transitioning Test from Critiquing to Defining

  • 2. ! ! ! ! !!!!! ! !!!! An independent software development consultant and trainer, Gerard Meszaros has more than thirty years of experience in software and more than a decade of experience in agile methods. Gerard is an expert in test automation patterns, refactoring of software and tests, and design for testability. He has applied automated unit and acceptance testing on projects ranging from full-on eXtreme Programming to traditional waterfall development in a wide range of industries. Gerard is an expert in the implementation and customization of agile methods such as Scrum, lean, and eXtreme Programming. His book xUnit Test Patterns—Refactoring Test Code won a Jolt Productivity Award in the Best Technical Book category. Gerard Meszaros Independent Consultant
  • 3. 1 1 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Build the Right Product Right Transitioning Test from Critiquing to Defining Gerard Meszaros Independent Consultant CTO of FeedXL.Com StarCanada2015@gerardm.com These slides: http://StarCanada2015.gerardm.com 2 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Product & I.T. I.T. Embedded Telecom My Background Gerard Meszaros StarCanada2015@gerardm.com • Software developer • Development manager • Project Manager • Software architect • OOA/OOD Mentor • Requirements (Use Case) Mentor • XP/TDD Mentor • Agile PM Mentor • Test Automation Consultant & Trainer • Lean/Agile Coach/Consultant
  • 4. 2 3 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Product & I.T. I.T. Embedded Telecom My Background • Software developer • Development manager • Project Manager • Software architect • OOA/OOD Mentor • Requirements (Use Case) Mentor • XP/TDD Mentor • Agile PM Mentor • Test Automation Consultant & Trainer • Lean/Agile Coach/Consultant Gerard Meszaros StarCanada2015@gerardm.com 4 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Agile Dev I’m From Development and I’m Here to Help
  • 5. 3 5 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Johnny’s Report Card http://fabulesslyfrugal.com/report-card-freebies-for-good-grades/ 6 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com QA as Quality Assessment
  • 6. 4 7 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Dev Test & Fix “Battleship” Development Quality Assessment Bug Reports Software Requirements B6! Missed! Requirements Test 8 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Requirements Development Testing
  • 7. 5 9 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Testing Development Requirements Testing Development Requirements 10 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Agile Test Problem Testing Development Requirements Testing Development Requirements Testing Development Requirements Testing Development Requirements
  • 8. 6 11 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Automated Testing is Essential http://www.cordistech.com/ 12 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com http://www.etfile.com/blog/to-automate-or-not-to-automate-that-is-the-question/
  • 9. 7 13 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com empowering http://www.slideshare.net/b2bSEO/robots-replacing-humans-automation 14 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Value of Automated Tests Tests as Documentation Faster regression testing Faster feedback on impact of changes Better coverage of data combinations Test with data without seeing that confidential data Provides more time for exploratory testing Can be done off hours; reduces delay/resources Writing tests helps us understand code better Know when problem was introduced Fast sanity checking Performance or load testing Setting up complex scenarios for manual/exploratory testing Unit/module tests improve modularity design and API’s Help Business understand what we/they are asking for Courtesy of participants of Tuesday’s tutorial
  • 10. 8 15 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Automated Tests are Just Code That Tests Other Code Who Tests This Code? Who Should Write This Code? 16 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey https://lmsa.site-ym.com/news/221726/Sex-and-Gender-Medical-Education-National-Student-Survey.htm
  • 11. 9 17 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey (Part 2) 18 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Audience Survey (Part 3) http://www.mcas.k12.in.us/Page/4826
  • 12. 10 19 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Insanity: •  Doing the same thing over and over again and expecting different results 20 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Project Retrospectives
  • 13. 11 21 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Prime Directive Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand 22 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Agile Retrospectives / Sprint Reviews
  • 14. 12 23 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Ineffective Retrospectives Stop Start More Actions 1. Do something 2. Do another thing 3. Do yet another thing 4. Do more stuff 5. Do yet more stuff 6. 24 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Ineffective Retrospectives Stop Start More Actions 1. Do something 2. Do another thing 3. Do yet another thing 4. Do more stuff 5. Do yet more stuff 6. How Do We Improve Retrospectives?
  • 15. 13 25 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com One Experiment per Iteration Stop Start More 1.  Hypothesis 2. Experiment 3. Analyse 4. Communicate 26 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com The Toyota Way
  • 16. 14 27 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s www.123rf.com 28 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Tests need to interact via the UI Test Automation is Hard! Product isn’t designed for testability Why? Why? Why? Tests are very complex
  • 17. 15 29 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? 30 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? QA is supposed to be “independent” Why?
  • 18. 16 31 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - Notation: + Increasing - Reducing 32 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - + + Notation: + Increasing - Reducing Reinforcing Cycle Balancing Cycle
  • 19. 17 33 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Systems Thinking A B + C - + + Notation: + Increasing - Reducing Vicious or Virtuous Cycle 34 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 5 Why’s of Test Automation Being Hard Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability Why? Why? Why? Why? Why? Why? Why? Why? Tests are very complex Why? Why? QA is supposed to be “independent” Why?
  • 20. 18 35 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test Automation – Systems Thinking Diagram Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability + + + + + + + + Tests are very complex + + QA is supposed to be “independent” + A Vicious Cycle 36 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Conway’s Law The Architecture of System will resemble the Organization that Built it. Development Team Test Team Business Team
  • 21. 19 37 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Resulting Architecture Specifi- cations Product Code Test Code Bus Test Dev 38 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Development flow Potentially Executable Specifications Example Interpreter Code Product Code Dev Test Dev DevTestBus Test
  • 22. 20 39 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Runtime View: Product Code Example Interpreter Code Report Card Potentially Executable Specifications Dev Test Dev DevTestBus Test 40 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Feedback Loops: Potentially Executable Specifications Product Code Example Interpreter Code Report Card What they say How they say it
  • 23. 21 41 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Development Feedback Loops: Potentially Executable Specifications Product Code Example Interpreter Code Report Card What they say How they say it Isn’t This “Just” Test Automation By Another Name? 42 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Tests vs. Checks vs. Examples?ref: http://www.satisfice.com/blog/archives/856
  • 24. 22 43 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Preparing Good Examples http://thepracticalleader.com/ 44 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example: Notifying of Bank Transactions Configure Notification Rules Suspend Notification Resume Notification Account Holder Process Transaction Transaction Settlement NotifybasedonchargeType NotifybasedonchargeContinent NotifybasedonchargeCountry NotifybasedonchargeCity&State SuspendNotification
  • 25. 23 45 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications •  Open MegaBank app •  Log in as “BobMa” with password ****** •  Click on “Manage Notifications” tab •  Click on “Add New Rule” button •  Select account “10035692877” •  Type “Default Rule” into field “rule name” •  Type “1000” into field “threshold amount” •  Click on “all transaction types” radio button •  Click on “all locations” radio button •  Click on “save changes” button •  … •  … 46 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications – 1/2 Given: User and Accounts Another attempt: When: Notification Rule is Configured Then: Notification Rule Configured and Active Then: Notification Rule is Active
  • 26. 24 47 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Checking Notifications – 2/2 Then: Expected Notifications When: The Transactions to be processed Another attempt: 48 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test Automation Pyramid Pyramid originally proposed by Mike Cohn Exploratory Tests Unit Tests Component Tests (API) System Tests (GUI)
  • 27. 25 49 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Unit Tests Component Tests (API) System Tests (GUI) Broad Narrow Detail HighLow Scope 50 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Broad Narrow Detail HighLow Scope Component Tests (API) System Tests (GUI)
  • 28. 26 51 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Behavior Specification at Right Level Component Tests (API) System Tests (GUI) Rules & Algorithms Multi- Use Case Workflows Incomplete Spec Too much detail Unmaintainable Detail HighLow Broad Narrow Scope Transactions (Use Cases) 52 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Too much detail Unmaintainable Changing Level of Abstraction/Detail Rules & Algorithms Multi- Use Case Workflows Transactions (Use Cases) Reduce Scope a Lot! ReduceDetailaLot Reduce Both a bit Incomplete Spec Broad Narrow Detail HighLow Scope
  • 29. 27 53 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example Given: User and Accounts When: Notification Rule is Configured Then: Notification Rule Configured and Active Then: Notification Rule is Active 54 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example “If it isn’t essential to conveying the essence of the behavior, it is essential to not include it.”
  • 30. 28 55 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Then: Expected Notifications When: The Transactions to be processed Refactoring Workflow Example 56 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example
  • 31. 29 57 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow Example 58 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Refactoring Workflow ExampleGiven: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications
  • 32. 30 59 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Given: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications credit credit 9991113333 Refactoring Workflow Example 60 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 9991113333 Refactoring Workflow Example Broad Scope (Multi-Actor); Minimum Detail (per Actor/Transaction); Given: User & Thresholds When: Transactions Are Processed Then: We Expect Notifications credit credit
  • 33. 31 61 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Filling in the Details Rules & Algorithms Multi- Use Case Workflows Transactions (Use Cases) Incomplete Spec ReduceDetailaLot Reduce Both a bit Too much detail Unmaintainable Reduce Scope a Lot! Broad Narrow Detail HighLow Scope 62 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Business Rule Example Process Transaction Threshold per Charge Type Configuration
  • 34. 32 63 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Process Transaction Threshold per Charge Type Narrow Scope (Single Rule) High Detail (Everything that matters) Example: Configuration Given these rules Then: The answer should be Business Rule Example When we ask NotificationRequired? with this transaction: 64 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Life Cycle of an Example User Goal Feature Story Title Story Narrative Story Scenarios Story Examples Executable Examples Satisfied Examples Formalize Automate Develop Product Make Concrete Add Data Define Acceptance Criteria Work Breakdown Product Design & Validation Elaborate
  • 35. 33 65 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Who Does Activities on an Example User Goal Feature Story Title Story Narrative Story Scenarios Story Examples Executable Examples Satisfied Examples Formalize Automate Develop Product Make Concrete Add Data Define Acceptance Criteria Work Breakdown Product Design & Validation Elaborate DevTest Bus DevTest Bus Dev Test Bus 66 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Test-After Architecture System Under Test Should we Notify?Process Transaction Configure Notification Threshold Notification Rules Transaction Interface Configuration User Interface Notification Log Workflow Test Do Notification. Bus Test Dev
  • 36. 34 67 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Example-Driven Architecture System Under Test Should we Notify? Do Notification. Process Transaction Configure Notification Threshold Notification Rules Notification Log Transaction Interface Configuration User Interface Workflow Example Workflow Keyword Interpreter Dev Dev Bus Test 68 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Transaction Interface Configuration User Interface Configure Notification Threshold System Under Test EDA – Business Rules Do Notification. Process Transaction Notification Rules Notification Log Notification Method Example Notification Rules Fake Notification Rule Example Should we Notify? Example Interpreter Example Interpreter Dev Bus Test Dev
  • 37. 35 69 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com EDA – Business Rules Should we Notify? Notification Rules Fake Notification Rule Example Example Interpreter Dev Bus Test Dev 70 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com After State – Back to Our Systems Diagram Dev doesn’t know how (or care) Tests need to interact via the UI Test Automation is Hard! It’s QA’s problem Tests are automated after Dev Complete QA isn’t involved in product definition QA is too busy testing last release Product isn’t designed for testability + + + + + + + + Tests are very complex + + QA is supposed to be “independent” + X X X X X XXX
  • 38. 36 71 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com After State – Back to Our Systems Diagram Dev knows how (and cares) Examples don’t need to interact via the UI Automated checking is Easy! Automation is Dev’s problem Examples are automated before Dev starts QA is involved in product definition QA has time for Examples and Exploratory testing Product is designed for testability + + + + + + + + Examples are simple + + A Virtuous Cycle 72 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com My Challenge to You Stop Start More 1.  Hypothesis 2. Experiment 3. Analyse 4. Communicate
  • 39. 37 73 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com 74 Copyright 2015 Gerard Meszaroshttp://StarCanada2015.gerardm.com Thank You! Gerard Meszaros StarCanada2015@gerardm.com http://www.xunitpatterns.com Slides: http://StarCanada2015.gerardm.com Call me when you: •  Want to transition to Agile or Lean •  Want to do Agile or Lean better •  Want to teach developers how to test •  Need help with test automation strategy •  Want to improve your test automation Jolt Productivity Award winner - Technical Books Available on MSDN: