Deep Testing, Deep Work - How and when we should enable deep work for testers
0 likes65 views
Not all testing is equal. Sometimes you are running a smoke check, testing the basic functionality of the game, and sometimes you are experimenting, investigating and trying to find the root cause of a problem, or exploring unknown territory. James Bach and Michael Bolton have talked about shallow and deep testing in the “Rapid Software Testing” namespace, saying that depth and shallowness are ways of talking about the thoroughness of testing. I think there is also an opportunity to connect deep and shallow testing to both the Cynefin framework, and to deep and shallow work, to draw some conclusions on how and when we should enable deep work for testers.
![Deep Testing, Deep Work
Introduction
Not all testing is equal. Sometimes you are running a smoke check, testing the basic
functionality of the game, and sometimes you are experimenting, investigating and trying to
find the root cause of a problem, or exploring unknown territory.
James Bach and Michael Bolton have talked about shallow and deep testing in the “Rapid
Software Testing” namespace, saying that depth and shallowness are ways of talking about
the thoroughness of testing [1][2][3].
I think there is also an opportunity to connect deep and shallow testing to both the Cynefin
framework [8], and to deep and shallow work [9][10][11], to draw some conclusions on how
and when we should enable deep work for testers.
Shallow and Deep Testing
In software testing we are often faced with a multitude of different types of test problems to
solve. Clear test problems, complicated test problems and complex test problems [6].
Clear [8] test problems can be basic types of tests, such as testing if the application starts,
and if basic functionality is working as expected. Cause and effect is relatively clear. This
could be defined as shallow tests. But this doesn’t say anything about the value of these tests
- they are extremely valuable and business critical, even though they are shallow [2][4][5].
Complicated [8] test problems have more dependencies and probably more steps, and may
require specific tools to run, but cause and effect is still relatively clear. This could potentially
still be done during shallow work, and with context-switching, but some focus may be
required depending on how complicated the problem is.
When we move into complex [8] tests, we move into a problem space where cause and
effect is unknown, and more exploration, investigation and experimentation is necessary.
This is where we utilise more deep testing. And require more deep work.
So our work often requires us to do a significant amount of shallow testing, but depending
on the context, also a significant amount of deep testing, where we are really utilising our
core test competence [6] of exploration, investigation and experimentation, and learning and
sharpening our skill set the most.](https://image.slidesharecdn.com/deeptestingdeepwork-250318142655-e3d2e4da/85/Deep-Testing-Deep-Work-How-and-when-we-should-enable-deep-work-for-testers-1-320.jpg)
![Deep Work for Deep Testing
So we have a complex test problem, and we want to solve that efficiently by doing some
deep testing. But we are constantly getting interrupted by meetings, instant messages, and
people coming over to ask questions, and it is costing us more than people think [10]. We
need to be able to concentrate and focus to be able to solve the problem efficiently. We
need deep work.
Deep work [9]:
“Professional activity performed in a state of distraction-free concentration that push your
cognitive capabilities to their limit. These efforts create new value, improve your skill, and are
hard to replicate.”
Shallow work [9]:
“Non-cognitively demanding, logistical-style tasks, often performed while distracted. These
efforts tend to not create new value in the world and are easy to replicate.”
You can see the similarities in definition between deep and shallow testing and deep and
shallow work. Deep work is what moves the needle in our work and our careers, but there is
also a limit to how much we can concentrate [11], and there is still a business critical demand
for shallow work in our field.
So how do we enable the deep work required to solve these complex test problems?
Enabling Deep Testing
There are many good examples of how to enable deep work [9][10]:
● Reduce amount of meetings to free up time for deep work
● Asynchronous communication - many meetings can be emails
● Schedule deep work sessions
● Create guidelines for emergencies that require interruptions
● Normalize saying no to requests when people are over capacity
● Flexible working hours
● Environments that are without distractions and are comfortable
● And so on …
Almost all of these enablers are the same for everyone in a development team - but the
structure and content of a deep work session will differ depending on the role. And we want
to understand what this could look like for a tester.](https://image.slidesharecdn.com/deeptestingdeepwork-250318142655-e3d2e4da/85/Deep-Testing-Deep-Work-How-and-when-we-should-enable-deep-work-for-testers-2-320.jpg)
![Deep work testing session
Luckily for us testers, James and Jonathan Bach already solved this problem for us more than
20 years ago, when they created session-based testing [7]. The perfect structure for deep
work and deep testing.
“An uninterrupted period of time spent testing, ideally lasting one to two hours. Each session
is focused on a charter, but testers can also explore new opportunities or issues during this
time. The tester creates and executes tests based on ideas, heuristics or whatever
frameworks to guide them and records their progress. This might be through the use of
written notes, video capture tools or by whatever method as deemed appropriate by the
tester.” [7]
And apparently we can at the most concentrate for up to 4 hours each day [11].
This means that we could for example use 4-6 hours for shallow work, and two 1-2 hour test
sessions for deep work per day. This would allow us to balance solving both clear test
problems, and do all the administrative tasks we need to perform, as well as experimenting,
investigating and exploring around those complex test problems that require us to be in a
deep work/deep testing mode.](https://image.slidesharecdn.com/deeptestingdeepwork-250318142655-e3d2e4da/85/Deep-Testing-Deep-Work-How-and-when-we-should-enable-deep-work-for-testers-3-320.jpg)
![References
[1] Testing deep and shallow
https://developsense.com/blog/2022/01/testing-deep-and-shallow
[2] Testing deep and shallow: Shallow is a feature, not an insult
https://developsense.com/blog/2022/01/testing-deep-and-shallow-2-shallow-is-not-an-insult
[3] Testing deep and shallow: Determination
https://developsense.com/blog/2022/11/testing-deep-and-shallow-3-determination
[4] Shallow testing gets a bad rap
https://www.annemariecharrett.com/bos-series-shallow-testing-gets-a-bad-wrap/
[5] Black Box testing
https://www.annemariecharrett.com/black-box-testing/
[6] Defining Test Competence
https://www.slideshare.net/slideshow/defining-test-competence-66709391/66709391
[7] Session-based testing
https://en.wikipedia.org/wiki/Session-based_testing
https://www.satisfice.com/download/session-based-test-management
[8] Cynefin framework
https://thecynefin.co/about-us/about-cynefin-framework/
[9] The Complete Guide to Deep Work
https://www.todoist.com/inspiration/deep-work
[10] Context-switching is the main productivity killer for developers
https://newsletter.techworld-with-milan.com/p/context-switching-is-the-main-productivity
[11] On team topologies and deep work
https://menzen.ski/posts/2025/02/15/on-team-topologies-and-deep-work/](https://image.slidesharecdn.com/deeptestingdeepwork-250318142655-e3d2e4da/85/Deep-Testing-Deep-Work-How-and-when-we-should-enable-deep-work-for-testers-4-320.jpg)
Deep Testing, Deep Work - How and when we should enable deep work for testers
- 1. Deep Testing, Deep Work Introduction Not all testing is equal. Sometimes you are running a smoke check, testing the basic functionality of the game, and sometimes you are experimenting, investigating and trying to find the root cause of a problem, or exploring unknown territory. James Bach and Michael Bolton have talked about shallow and deep testing in the “Rapid Software Testing” namespace, saying that depth and shallowness are ways of talking about the thoroughness of testing [1][2][3]. I think there is also an opportunity to connect deep and shallow testing to both the Cynefin framework [8], and to deep and shallow work [9][10][11], to draw some conclusions on how and when we should enable deep work for testers. Shallow and Deep Testing In software testing we are often faced with a multitude of different types of test problems to solve. Clear test problems, complicated test problems and complex test problems [6]. Clear [8] test problems can be basic types of tests, such as testing if the application starts, and if basic functionality is working as expected. Cause and effect is relatively clear. This could be defined as shallow tests. But this doesn’t say anything about the value of these tests - they are extremely valuable and business critical, even though they are shallow [2][4][5]. Complicated [8] test problems have more dependencies and probably more steps, and may require specific tools to run, but cause and effect is still relatively clear. This could potentially still be done during shallow work, and with context-switching, but some focus may be required depending on how complicated the problem is. When we move into complex [8] tests, we move into a problem space where cause and effect is unknown, and more exploration, investigation and experimentation is necessary. This is where we utilise more deep testing. And require more deep work. So our work often requires us to do a significant amount of shallow testing, but depending on the context, also a significant amount of deep testing, where we are really utilising our core test competence [6] of exploration, investigation and experimentation, and learning and sharpening our skill set the most.
- 2. Deep Work for Deep Testing So we have a complex test problem, and we want to solve that efficiently by doing some deep testing. But we are constantly getting interrupted by meetings, instant messages, and people coming over to ask questions, and it is costing us more than people think [10]. We need to be able to concentrate and focus to be able to solve the problem efficiently. We need deep work. Deep work [9]: “Professional activity performed in a state of distraction-free concentration that push your cognitive capabilities to their limit. These efforts create new value, improve your skill, and are hard to replicate.” Shallow work [9]: “Non-cognitively demanding, logistical-style tasks, often performed while distracted. These efforts tend to not create new value in the world and are easy to replicate.” You can see the similarities in definition between deep and shallow testing and deep and shallow work. Deep work is what moves the needle in our work and our careers, but there is also a limit to how much we can concentrate [11], and there is still a business critical demand for shallow work in our field. So how do we enable the deep work required to solve these complex test problems? Enabling Deep Testing There are many good examples of how to enable deep work [9][10]: ● Reduce amount of meetings to free up time for deep work ● Asynchronous communication - many meetings can be emails ● Schedule deep work sessions ● Create guidelines for emergencies that require interruptions ● Normalize saying no to requests when people are over capacity ● Flexible working hours ● Environments that are without distractions and are comfortable ● And so on … Almost all of these enablers are the same for everyone in a development team - but the structure and content of a deep work session will differ depending on the role. And we want to understand what this could look like for a tester.
- 3. Deep work testing session Luckily for us testers, James and Jonathan Bach already solved this problem for us more than 20 years ago, when they created session-based testing [7]. The perfect structure for deep work and deep testing. “An uninterrupted period of time spent testing, ideally lasting one to two hours. Each session is focused on a charter, but testers can also explore new opportunities or issues during this time. The tester creates and executes tests based on ideas, heuristics or whatever frameworks to guide them and records their progress. This might be through the use of written notes, video capture tools or by whatever method as deemed appropriate by the tester.” [7] And apparently we can at the most concentrate for up to 4 hours each day [11]. This means that we could for example use 4-6 hours for shallow work, and two 1-2 hour test sessions for deep work per day. This would allow us to balance solving both clear test problems, and do all the administrative tasks we need to perform, as well as experimenting, investigating and exploring around those complex test problems that require us to be in a deep work/deep testing mode.
- 4. References [1] Testing deep and shallow https://developsense.com/blog/2022/01/testing-deep-and-shallow [2] Testing deep and shallow: Shallow is a feature, not an insult https://developsense.com/blog/2022/01/testing-deep-and-shallow-2-shallow-is-not-an-insult [3] Testing deep and shallow: Determination https://developsense.com/blog/2022/11/testing-deep-and-shallow-3-determination [4] Shallow testing gets a bad rap https://www.annemariecharrett.com/bos-series-shallow-testing-gets-a-bad-wrap/ [5] Black Box testing https://www.annemariecharrett.com/black-box-testing/ [6] Defining Test Competence https://www.slideshare.net/slideshow/defining-test-competence-66709391/66709391 [7] Session-based testing https://en.wikipedia.org/wiki/Session-based_testing https://www.satisfice.com/download/session-based-test-management [8] Cynefin framework https://thecynefin.co/about-us/about-cynefin-framework/ [9] The Complete Guide to Deep Work https://www.todoist.com/inspiration/deep-work [10] Context-switching is the main productivity killer for developers https://newsletter.techworld-with-milan.com/p/context-switching-is-the-main-productivity [11] On team topologies and deep work https://menzen.ski/posts/2025/02/15/on-team-topologies-and-deep-work/