A business case for a modern QA organization
0 likes29 views
The document discusses the evolution of quality assurance (QA) organizations from a traditional model to a modern approach, highlighting key differences such as team integration, testing focus, and collaboration. Traditional QA relied on large, separate teams with a low developer testing engagement, while the modern model emphasizes smaller, integrated teams where QA specialists work closely with developers throughout the software development lifecycle. This shift addresses problems of high operational costs, long lead times, and insufficient quality by fostering accountability, incorporating quality at every stage, and utilizing specialized QA expertise.
![Modern QA Organization Traits
1:7 QA Specialist:Developer ratio - basically one QA Specialist in an Agile team
QA Specialists are an integrated part of Agile teams
Focus on testing, not test cases - test cases are just artifacts that may or may not be useful
depending on context
We know that there is an unlimited number of tests we could run at any given time, and we let
risk analysis and opportunity cost guide how we spend our time efficiently to maximize value
Developers perform simple[1] and complicated[1] testing, QA Specialist focus on complex[1]
testing, and testing which require specialist competence and tools
Extensive collaboration between developers and testers, supporting and coaching each other
with their specialist competence
Tests are not designed in isolation, they are designed as an integrated part of the
development process
Test and quality is part of the entire SDLC, both Shift Left [2] and Shift Right [2]
Testers uncover quality information/intelligence which stakeholders use to make decision with
regards to quality, and the team as a whole owns the quality of the product
Test automation is software development and is driven by someone with that expertise
The more insight you have into the product as a QA Specialist the better - using the product
without any insight can be organized as specific test activities, such as user testing or
persona testing [3]
Being a QA Specialist requires a specific skill set that is not easily replaceable, just as a
developer or designer
QA Specialists use their expertise to influence good quality practices and processes
So why is the modern approach more effective and efficient than what was common 20 years
ago? What are the problems that were solved with this modern approach, and how were they
solved?](https://image.slidesharecdn.com/abusinesscaseforamodernqaorganization1-230531124243-24d7d149/85/A-business-case-for-a-modern-QA-organization-2-320.jpg)
![We can boil it down to three major problems:
● High operational costs
● Long lead times
● Insufficient quality
1. High Operational Costs
Having a large test department carries a high operational cost, and unless that cost actually
provides value in the form of shorter lead times and higher quality, it becomes a problem.
The solution to this is simple. Don’t have a large test department unless it provides value.
2. Long lead times
What caused long lead times with the traditional approach?
● Bugs and problems being found late in the SDLC, requiring delays to fix
● Handovers between development teams and test teams, especially when tasks go back
and forth due to lack of collaboration
● Redundant testing
● Creating unnecessary artifacts, such as test cases and test reports
● Test and QA activities are not planned accordingly due to lack of communication and
expertise in the right place
This is mainly solved in the modern approach by:
● Integrating QA specialists into development teams reduces handovers
● Working with quality and test throughout the SDLC reduces late bugs and problems, and
is planned accordingly from the start
● Risk-based testing [4] reduces redundant testing
● QA Specialists integrated in teams reduces unnecessary reporting, and focus on testing
instead of test cases reduces unnecessary test case creation
3. Insufficient quality
What caused insufficient quality with the traditional approach?
● Quality cannot be tested in, it needs to be built in from the start, and be on everyone’s
minds throughout the SDLC](https://image.slidesharecdn.com/abusinesscaseforamodernqaorganization1-230531124243-24d7d149/85/A-business-case-for-a-modern-QA-organization-3-320.jpg)
![● Developers not owning quality causes lack of agency in developing high quality code
and not designing around testability
● Lack of collaboration and coaching between people with different skill sets - QA
Specialists and Developers
● Making test automation a problem to solve for people without the right expertise
● Leaving business decisions related to quality in the hands of QA Specialists, who do not
have the right expertise to make those decisions
● Not hiring experienced QA Specialists with valuable expertise and skill set
● Not using QA Specialists’ expertise where it adds the most value - solving complex test
problems, but instead forcing QA Specialists to spend most of their time handling simple
[1] test problems that could be done by anyone
● Testing without actually understanding the underlying system can easily miss
non-intuitive test scenarios at worst, and just be inefficient at best
● Processes have been defined without QA expertise
This is mainly solved in the modern approach by:
● Quality and test are taken into consideration throughout the entire SLDC
● QA Specialists and developers work closely together, supporting and coaching each
other, taking testing and QA to the next level
● Everyone is accountable for their work, and the team as a whole is accountable for the
quality of the product
● Test automation is driven by someone with software development expertise, in
collaboration with anyone who can provide value, often QA Specialists
● QA Specialists are hired because of their specialist skills, just like developers are
● QA Specialists focus on solving complex test problems, with a strong understanding of
the underlying system, while everyone is responsible for testing their deliverables to the
best of their abilities
● QA Specialists use their expertise to influence practices and processes to achieve higher
quality and efficiency
● Business decisions around quality, such as what bugs to fix, are taken by people with the
right business expertise](https://image.slidesharecdn.com/abusinesscaseforamodernqaorganization1-230531124243-24d7d149/85/A-business-case-for-a-modern-QA-organization-4-320.jpg)
![Comparison Chart
Traditional QA Organization Traits Modern QA Organization Traits
Almost as large as the development organization - 1:1
or 1:2 Tester:Developer ratio
1:7 Developer:QA Specialist ratio - basically one QA
Specialist in an Agile team
Separate development and test teams QA Specialists are an integrated part of Agile teams
Focus on test cases for traceability, visibility and
reporting
Focus on testing, not test cases - test cases are just
artifacts that may or may not be useful depending on
context
We run all our tests as often as we can because if they
all pass we know everything is working
We know that there is an unlimited number of tests we
could run at any given time, and we let risk analysis and
opportunity cost guide how we spend our time efficiently
to maximize value
Almost non-existent testing by developers, because
they should focus their time on development where they
add the most value
Developers perform simple[1] and complicated[1]
testing, QA Specialist focus on complex[1] testing, and
testing which require specialist competence and tools
Little collaboration between developers and testers until
there is something to test
Extensive collaboration between developers and
testers, supporting and coaching each other with their
specialist competence
Test cases are sometimes designed by a Test Designer
and then executed by a tester
Tests are not designed in isolation, they are designed
as an integrated part of the development process
Testing is performed very late in the SDLC, when the
software is “done”
Test and quality is part of the entire SDLC, both Shift
Left [2] and Shift Right [2]
Testers are gatekeepers of quality and push developers
to fix bugs
Testers uncover quality information/intelligence which
stakeholders use to make decision with regards to
quality, and the team as a whole owns the quality of the
product
Test automation is driven by the QA organization
because it only impacts their work
Test automation is software development and is driven
by someone with that expertise
Focus on Black Box testing because testers should
emulate our customers
The more insight you have into the product as a QA
Specialist the better - using the product without any
insight can be organized as specific test activities, such
as user testing or persona testing [3]
Being a tester is a low skilled and low value job, and
you are easily replaceable
Being a QA Specialist requires a specific skill set that is
not easily replaceable, just as a developer or designer
Testers adapt around development processes QA Specialists use their expertise to influence good
quality practices and processes](https://image.slidesharecdn.com/abusinesscaseforamodernqaorganization1-230531124243-24d7d149/85/A-business-case-for-a-modern-QA-organization-5-320.jpg)
![References
[1] Cynefin Framework
https://en.wikipedia.org/wiki/Cynefin_framework
[2] Shift Left, Shift Right
https://www.dynatrace.com/news/blog/what-is-shift-left-and-what-is-shift-right/
[3] Personas - A Simple Introduction
https://www.interaction-design.org/literature/article/personas-why-and-how-you-should-use-them
[4] Heuristic Risk-Based Software Testing
https://www.satisfice.com/download/heuristic-risk-based-software-testing](https://image.slidesharecdn.com/abusinesscaseforamodernqaorganization1-230531124243-24d7d149/85/A-business-case-for-a-modern-QA-organization-6-320.jpg)
A business case for a modern QA organization
- 1. A Business Case for a Modern QA Organization To describe what a modern QA organization could look like, we must first understand our history. What defined a QA organization some 10-20 years ago? Traditional QA Organization Traits Almost as large as the development organization - 1:1 or 1:2 Tester:Developer ratio Separate development and test teams Focus on test cases for traceability, visibility and reporting We run all our tests as often as we can because if they all pass we know everything is working Almost non-existent testing by developers, because they should focus their time on development where they add the most value Little collaboration between developers and testers until there is something to test Test cases are sometimes designed by a Test Designer and then executed by a tester Testing is performed very late in the SDLC, when the software is “done” Testers are gatekeepers of quality and push developers to fix bugs Test automation is driven by the QA organization because it only impacts their work Focus on Black Box testing because testers should emulate our customers Being a tester is a low skilled and low value job, and you are easily replaceable Testers adapt around development processes This was the reality around 20 years ago, but software development and testing has luckily evolved since then. Before we go into the details of why this was not efficient or effective, let’s look at an overview of what a modern QA organization could look like in comparison.
- 2. Modern QA Organization Traits 1:7 QA Specialist:Developer ratio - basically one QA Specialist in an Agile team QA Specialists are an integrated part of Agile teams Focus on testing, not test cases - test cases are just artifacts that may or may not be useful depending on context We know that there is an unlimited number of tests we could run at any given time, and we let risk analysis and opportunity cost guide how we spend our time efficiently to maximize value Developers perform simple[1] and complicated[1] testing, QA Specialist focus on complex[1] testing, and testing which require specialist competence and tools Extensive collaboration between developers and testers, supporting and coaching each other with their specialist competence Tests are not designed in isolation, they are designed as an integrated part of the development process Test and quality is part of the entire SDLC, both Shift Left [2] and Shift Right [2] Testers uncover quality information/intelligence which stakeholders use to make decision with regards to quality, and the team as a whole owns the quality of the product Test automation is software development and is driven by someone with that expertise The more insight you have into the product as a QA Specialist the better - using the product without any insight can be organized as specific test activities, such as user testing or persona testing [3] Being a QA Specialist requires a specific skill set that is not easily replaceable, just as a developer or designer QA Specialists use their expertise to influence good quality practices and processes So why is the modern approach more effective and efficient than what was common 20 years ago? What are the problems that were solved with this modern approach, and how were they solved?
- 3. We can boil it down to three major problems: ● High operational costs ● Long lead times ● Insufficient quality 1. High Operational Costs Having a large test department carries a high operational cost, and unless that cost actually provides value in the form of shorter lead times and higher quality, it becomes a problem. The solution to this is simple. Don’t have a large test department unless it provides value. 2. Long lead times What caused long lead times with the traditional approach? ● Bugs and problems being found late in the SDLC, requiring delays to fix ● Handovers between development teams and test teams, especially when tasks go back and forth due to lack of collaboration ● Redundant testing ● Creating unnecessary artifacts, such as test cases and test reports ● Test and QA activities are not planned accordingly due to lack of communication and expertise in the right place This is mainly solved in the modern approach by: ● Integrating QA specialists into development teams reduces handovers ● Working with quality and test throughout the SDLC reduces late bugs and problems, and is planned accordingly from the start ● Risk-based testing [4] reduces redundant testing ● QA Specialists integrated in teams reduces unnecessary reporting, and focus on testing instead of test cases reduces unnecessary test case creation 3. Insufficient quality What caused insufficient quality with the traditional approach? ● Quality cannot be tested in, it needs to be built in from the start, and be on everyone’s minds throughout the SDLC
- 4. ● Developers not owning quality causes lack of agency in developing high quality code and not designing around testability ● Lack of collaboration and coaching between people with different skill sets - QA Specialists and Developers ● Making test automation a problem to solve for people without the right expertise ● Leaving business decisions related to quality in the hands of QA Specialists, who do not have the right expertise to make those decisions ● Not hiring experienced QA Specialists with valuable expertise and skill set ● Not using QA Specialists’ expertise where it adds the most value - solving complex test problems, but instead forcing QA Specialists to spend most of their time handling simple [1] test problems that could be done by anyone ● Testing without actually understanding the underlying system can easily miss non-intuitive test scenarios at worst, and just be inefficient at best ● Processes have been defined without QA expertise This is mainly solved in the modern approach by: ● Quality and test are taken into consideration throughout the entire SLDC ● QA Specialists and developers work closely together, supporting and coaching each other, taking testing and QA to the next level ● Everyone is accountable for their work, and the team as a whole is accountable for the quality of the product ● Test automation is driven by someone with software development expertise, in collaboration with anyone who can provide value, often QA Specialists ● QA Specialists are hired because of their specialist skills, just like developers are ● QA Specialists focus on solving complex test problems, with a strong understanding of the underlying system, while everyone is responsible for testing their deliverables to the best of their abilities ● QA Specialists use their expertise to influence practices and processes to achieve higher quality and efficiency ● Business decisions around quality, such as what bugs to fix, are taken by people with the right business expertise
- 5. Comparison Chart Traditional QA Organization Traits Modern QA Organization Traits Almost as large as the development organization - 1:1 or 1:2 Tester:Developer ratio 1:7 Developer:QA Specialist ratio - basically one QA Specialist in an Agile team Separate development and test teams QA Specialists are an integrated part of Agile teams Focus on test cases for traceability, visibility and reporting Focus on testing, not test cases - test cases are just artifacts that may or may not be useful depending on context We run all our tests as often as we can because if they all pass we know everything is working We know that there is an unlimited number of tests we could run at any given time, and we let risk analysis and opportunity cost guide how we spend our time efficiently to maximize value Almost non-existent testing by developers, because they should focus their time on development where they add the most value Developers perform simple[1] and complicated[1] testing, QA Specialist focus on complex[1] testing, and testing which require specialist competence and tools Little collaboration between developers and testers until there is something to test Extensive collaboration between developers and testers, supporting and coaching each other with their specialist competence Test cases are sometimes designed by a Test Designer and then executed by a tester Tests are not designed in isolation, they are designed as an integrated part of the development process Testing is performed very late in the SDLC, when the software is “done” Test and quality is part of the entire SDLC, both Shift Left [2] and Shift Right [2] Testers are gatekeepers of quality and push developers to fix bugs Testers uncover quality information/intelligence which stakeholders use to make decision with regards to quality, and the team as a whole owns the quality of the product Test automation is driven by the QA organization because it only impacts their work Test automation is software development and is driven by someone with that expertise Focus on Black Box testing because testers should emulate our customers The more insight you have into the product as a QA Specialist the better - using the product without any insight can be organized as specific test activities, such as user testing or persona testing [3] Being a tester is a low skilled and low value job, and you are easily replaceable Being a QA Specialist requires a specific skill set that is not easily replaceable, just as a developer or designer Testers adapt around development processes QA Specialists use their expertise to influence good quality practices and processes
- 6. References [1] Cynefin Framework https://en.wikipedia.org/wiki/Cynefin_framework [2] Shift Left, Shift Right https://www.dynatrace.com/news/blog/what-is-shift-left-and-what-is-shift-right/ [3] Personas - A Simple Introduction https://www.interaction-design.org/literature/article/personas-why-and-how-you-should-use-them [4] Heuristic Risk-Based Software Testing https://www.satisfice.com/download/heuristic-risk-based-software-testing