![PERSPECTIVE
FROM A TESTER
JONATHAN BACH takes his job as Test Lead for ership made him want his apps tested thoroughly. “When that
Microsoft’s Systems Management Server team seriously. But he product was delivered to the customers, they thought it was re-
describes his job at the software giant as pure play. “I love the ally cool, and I wanted to be able to take the credit for a solid ap-
diamond in the rough,” says Bach. “Testing is like hide and seek, plication. Good testing helped me do that.”
scavenger hunting, taking the metal detector onto the beach.” Now back in his full-time tester role, Bach depends on the
From his perspective, testing is a treasure expedition, along with developer to help him build that same kind of partnership. “I exist
all the inherent tedium and risks. “Just like prospecting for to make their code better,” he stresses, “to be their bodyguard,
gold,” he says, “you’re going to come up with a lot of useless their advocate, as well as the customer’s.” Before the developer’s
dirt.” The payoff, for Bach and his fellow testers, is finding that code leaves the building, Bach says his team wants to do every-
one thing buried in the rock that will prevent the customer from thing possible to make sure no one is going to be embarrassed by
ever calling up Product Support. the quality of the product.
That, says Bach, is something developers don’t always un- The development team can help, Bach says, by helping steer
derstand. “Testers don’t set out to break developers’ software,” he testers toward any potential danger zones in the code. “You [the
says. “As Cem Kaner has said, the software comes to us already developer] know the code better than anyone else,” he says. “It’s
broken. We just want to find the breaks that are already there.” great when a developer can feel okay about telling me ‘Here’s
He remembers what it’s like to work on the development where the risk is in my code’ or ‘This is new code.’ ”
side, though—to create something out of nothing, and then have Bach says that developing good relationships between
to hand over your child for critique. “When I was writing test testers and developers is vital if that kind of sharing is going to
apps for Visual Basic a few years ago,” Bach remembers, “I had happen. “If we both understand that synergy, that two people
the same feelings when the time came to hand my work over to with totally different perspectives will always find different
my tester.” He understands the potential hit to the ego at that bugs,” Bach says, “then we’ll invest in each other to take ad-
stage of the reviews, but also remembers how his pride of own- vantage of that resource.” —A.W.
FROM A DEVELOPER
TOM FLAHERTY admits it: If software development valiant effort, he says, but sometimes developers’ efforts to em-
were left solely in the hands of the developers, applications pathize with the unknown key tapper can only go so far. You do
would be pretty much as they were twenty-five years ago. know the product, and your brain travels in those familiar paths.
“Not in terms of what functions they perform,” he’s quick to “The testers can find things in a product you weren’t even
clarify, “but in the sense that only people whose vocation is com- aware of.”
puter technology would be able to use our products.” And that, he Flaherty depends on his QA team members for other rea-
says, is just part of the way he and his fellow developers natural- sons, too. “Sometimes,” he says, “we developers walk a very fine
ly look at software. “We sit there all day thinking about the in- line in how much time and attention we can spend thinking about
sides of an application,” says Flaherty. “Unless it’s a piece of soft- bugs.” Do only a brief look for functionality showstoppers before
ware for which we’ll end up being the primary users, we think handing code off to the testers and the testers grumble. Spend a
more about what the inner mechanics of function will make pos- week of your development time investigating one bug and your
sible—not about the million different ways an end user on the manager is unhappy. “I work out where that line is and rely on a
outside might try to extract that functionality.” Developers depend good testing team to do what needs to be done,” Flaherty says.
on testers’ abilities to look at the product the way the real world “Better to let the experts spend time on bug-finding because
will see it, manipulating it from a user’s point of view. testers are usually faster, more efficient, and better at finding
Throughout his eighteen years of programming, Flaherty those elusive bugs that, for us, are not reproducible, no matter
has relied on his testers’ unique perspectives to balance his own how many times we try.” Testers’ abilities to find those bugs
approach. “I don’t have hours to play with each part of the prod- through repetitive testing earn them Flaherty’s admiration.
uct,” he says, “trying to pretend like I don’t know how it works, “Thank goodness a tester’s mind works like that; to do that, to fo-
trying to overcome my biases about how things should work or cus after the thirty-ninth run of the same test…would drive most
how users should interact with my code.” You can make a developers out of their trees.” —A.W.
43
Januar y/Februar y 2000 Software Testing & Quality Engineering www.stqemagazine.com](https://image.slidesharecdn.com/testersanddevelopersthinkdifferently-120424060231-phpapp02/85/Testers-and-developers-think-differently-2-320.jpg)
Testers and developers think differently
- 1. Management & Teams Testers and Developers Think Differently Understanding and utilizing the diverse traits of key players on your team by Bret Pettichord S oftware development requires various tal- ents and roles. This article takes a look at two of these roles: testers and developers. On an effective team, testers and developers complement one another, each providing perspectives and skills that the other may lack. A common mistake is to see testers as junior developers, and to thus encourage them to emulate the skills and attitudes of the developers. In fact, good able to make judgements even when they may not have mas- tered the specific subject at hand. Their strength is often their ability to be generalists; they need to have a broad un- derstanding of many areas. This contrasts with developers, who are often required to be specialists in particular techni- cal areas (networking protocols, for example, or database in- ternals or display libraries). Testers, in contrast, need to be able to get up to speed quickly on any product that they test. testers have many traits that directly contrast with the They often have little time to learn a new product or feature. traits good developers need. With understanding, man- On the other hand, developers need to have a thorough agers can make the most of these divergent skill sets and understanding of the libraries or protocols they’ll be using use them to build a successful team. before they can work on something new. Otherwise, they Many developers don’t realize how difficult system may break something. They’re used to being allowed to gain testing can be. It takes patience and flexibility. It requires mastery before being expected to deliver. On my project, I an eye for detail and an understanding of the big picture. saw developers who were used to this kind of arrangement Many testers are frustrated working with developers who really struggle when they had to test dozens of features that think testing is easy. My understanding of this dynamic were new to them in a short period of time. was deepened when I got a chance to see several develop- Testers need to have the kind of knowledge that users ers take over the system testing. They had been used to have. This helps them use the product the way a user would, relying on a team of dedicated testers, who were no instead of the way the developer might like them to. Thus it longer available to test the product. can be important for them to not be familiar with the internal Since I had experience testing the product, I trained architecture of the product or perhaps even to act as though them during this project and saw them struggle with tasks they don’t know it when testing. The tester has to value a cer- they had previously underestimated. In the process, I no- tain kind of ignorance or naivete. Developers, of course, ticed some differences between the skills and attitudes of can’t afford this kind of ignorance. Developers may see this testers and developers. kind of ignorance as a sign that a tester isn’t smart enough I want to avoid stereotyping. There are, of course, many or is even being difficult. I had great difficulty getting devel- good developers who can test well, and there are many opers to test from a user’s perspective and put aside the testers who can write decent code. But in many ways, the knowledge that they had about the product internals. skills that the jobs require are different and often in opposi- Good testers are dilettantes. They need to know tion to each other. Understanding this is important to get- something about the customer’s domain, as well as infor- ting teams of different people to work well together. mation about computer systems. They need to be good at gaining superficial knowledge. They may even have to re- sist becoming too knowledgeable—or at least be able to Embracing “Dilettantism” pretend that they don’t know things that they really do. A dilettante is someone who dabbles in a practice or study Developers need to see the value in this. This doesn’t indi- without gaining mastery. The term is often used to disparage cate a lack of intellectual ability—it’s just a different atti- a lack of commitment or seriousness, but good testers are tude about acquiring knowledge. continued 42 www.stqemagazine.com Software Testing & Quality Engineering Januar y/Februar y 2000
- 2. PERSPECTIVE FROM A TESTER JONATHAN BACH takes his job as Test Lead for ership made him want his apps tested thoroughly. “When that Microsoft’s Systems Management Server team seriously. But he product was delivered to the customers, they thought it was re- describes his job at the software giant as pure play. “I love the ally cool, and I wanted to be able to take the credit for a solid ap- diamond in the rough,” says Bach. “Testing is like hide and seek, plication. Good testing helped me do that.” scavenger hunting, taking the metal detector onto the beach.” Now back in his full-time tester role, Bach depends on the From his perspective, testing is a treasure expedition, along with developer to help him build that same kind of partnership. “I exist all the inherent tedium and risks. “Just like prospecting for to make their code better,” he stresses, “to be their bodyguard, gold,” he says, “you’re going to come up with a lot of useless their advocate, as well as the customer’s.” Before the developer’s dirt.” The payoff, for Bach and his fellow testers, is finding that code leaves the building, Bach says his team wants to do every- one thing buried in the rock that will prevent the customer from thing possible to make sure no one is going to be embarrassed by ever calling up Product Support. the quality of the product. That, says Bach, is something developers don’t always un- The development team can help, Bach says, by helping steer derstand. “Testers don’t set out to break developers’ software,” he testers toward any potential danger zones in the code. “You [the says. “As Cem Kaner has said, the software comes to us already developer] know the code better than anyone else,” he says. “It’s broken. We just want to find the breaks that are already there.” great when a developer can feel okay about telling me ‘Here’s He remembers what it’s like to work on the development where the risk is in my code’ or ‘This is new code.’ ” side, though—to create something out of nothing, and then have Bach says that developing good relationships between to hand over your child for critique. “When I was writing test testers and developers is vital if that kind of sharing is going to apps for Visual Basic a few years ago,” Bach remembers, “I had happen. “If we both understand that synergy, that two people the same feelings when the time came to hand my work over to with totally different perspectives will always find different my tester.” He understands the potential hit to the ego at that bugs,” Bach says, “then we’ll invest in each other to take ad- stage of the reviews, but also remembers how his pride of own- vantage of that resource.” —A.W. FROM A DEVELOPER TOM FLAHERTY admits it: If software development valiant effort, he says, but sometimes developers’ efforts to em- were left solely in the hands of the developers, applications pathize with the unknown key tapper can only go so far. You do would be pretty much as they were twenty-five years ago. know the product, and your brain travels in those familiar paths. “Not in terms of what functions they perform,” he’s quick to “The testers can find things in a product you weren’t even clarify, “but in the sense that only people whose vocation is com- aware of.” puter technology would be able to use our products.” And that, he Flaherty depends on his QA team members for other rea- says, is just part of the way he and his fellow developers natural- sons, too. “Sometimes,” he says, “we developers walk a very fine ly look at software. “We sit there all day thinking about the in- line in how much time and attention we can spend thinking about sides of an application,” says Flaherty. “Unless it’s a piece of soft- bugs.” Do only a brief look for functionality showstoppers before ware for which we’ll end up being the primary users, we think handing code off to the testers and the testers grumble. Spend a more about what the inner mechanics of function will make pos- week of your development time investigating one bug and your sible—not about the million different ways an end user on the manager is unhappy. “I work out where that line is and rely on a outside might try to extract that functionality.” Developers depend good testing team to do what needs to be done,” Flaherty says. on testers’ abilities to look at the product the way the real world “Better to let the experts spend time on bug-finding because will see it, manipulating it from a user’s point of view. testers are usually faster, more efficient, and better at finding Throughout his eighteen years of programming, Flaherty those elusive bugs that, for us, are not reproducible, no matter has relied on his testers’ unique perspectives to balance his own how many times we try.” Testers’ abilities to find those bugs approach. “I don’t have hours to play with each part of the prod- through repetitive testing earn them Flaherty’s admiration. uct,” he says, “trying to pretend like I don’t know how it works, “Thank goodness a tester’s mind works like that; to do that, to fo- trying to overcome my biases about how things should work or cus after the thirty-ninth run of the same test…would drive most how users should interact with my code.” You can make a developers out of their trees.” —A.W. 43 Januar y/Februar y 2000 Software Testing & Quality Engineering www.stqemagazine.com
- 3. GOOD TESTERS system design perspective even when they covered dis- Get up to speed quickly tinct user scenarios. Domain knowledge It also showed up in analyzing test failures. On a cou- Ignorance is important ple of occasions, a lot of time was spent tracking down what I perceived to be minor anomalies. Although the GOOD DEVELOPERS software behaved as a user would reasonably expect, the Thorough understanding results were not what the system design would lead one to Knowledge of product internals expect—so the developers were naturally driven to ana- Expertise is important lyze them. Testers need to be encouraged to understand the cus- tomers’ needs. When testers are recruited from customer Modeling User Behavior support this happens automatically; it also happens when Having good models of the users’ behavior—and of the testers are recruited from actual customers or from the system design—is extremely important to software devel- targeted user community. Don’t let your testers be treated opment. Good developers and testers need both. But there simply as developers’ assistants. Make sure that they get are times when each discipline stresses different talents exposure to customers—and to the people at your compa- and calls for different emphases. As system designs be- ny who work directly with the customers. come more complicated, developers have to spend more and more time and effort making sure that they under- GOOD TESTERS stand the designs. At those times, having testers take re- Model user behavior sponsibility for understanding user needs and behavior Focus on what can go wrong can be very helpful to making sure that the product is Focus on severity of problem tested in realistic scenarios. A good example of this is error testing. Good testers GOOD DEVELOPERS know that users make numerous errors when learning and Model system design using new products. We all do this. More people prefer Focus on how it can work learning by trying and seeing what happens than by read- Focus on interest in problem ing a manual. Thus it is important to test software with small errors, typos, and nonsensical requests. Tests need to make sure that users can change their mind and undo partially completed work. On the other hand, the complex- Thinking Empirically ity of software products often requires developers to con- Good testing is governed by the scientific model. The “theo- centrate more on what needs to happen to fulfill a well-de- ry” being tested is that the software works. Testers design fined request. To do that, they need to focus on the system experiments, as Kaner says in Testing Computer Soft- design. ware, to see if they can falsify the “theory.” Good testers This difference in focus can also be seen in typical re- know how to design experiments, and they often benefit sponses to bugs. Testers will often assess a problem in from previous study of the sciences. Good testers think em- terms of the potential severity for a customer. This is pirically, in terms of observed behavior. done, perhaps tacitly, by having an intuitive model of how Developing software, on the other hand, is much like customers will use the product. How likely will customers creating theories. Laws are specified and rules are ap- encounter the bug? What might the consequences be? plied. Many developers have benefited from studying Developers focusing on system design may respond mathematics. Good developers think theoretically. to bugs in one of two ways. First, they may dismiss a Developers who are focusing on their theory of how problem. “Why would anyone want to do that?” On the the software works might dismiss a bug report that de- other hand, they may see the bug as an interesting prob- scribes behavior not allowed by their software theory. lem even if its effect on functionality is not that important “That can’t happen; it’s not possible.” Good testers focus to customers. In these cases, the developers are primarily on what actually happens, and like other experimenters, working from their model of the system behavior. In the keep detailed logs. “Well, it did happen; here are the cir- first case, the software is working in accordance with the cumstances.” People who are used to thinking theoretically design, so they are dismissive. It works as designed. In the have a hard time accepting aberrant behavior without second, the problem may show a flaw in the design or at some kind of explanation. Good testers are skeptics, least in their understanding of the design. Of course, it’s whereas good developers are believers. possible that this small consequence indicates a serious In my project, I had trouble justifying to some develop- design flaw. This is why it’s wise to encourage developers ers why we considered certain tests necessary. In these cas- to investigate these kinds of problems. Having different es, they wanted some reason for doubting that the software people take different approaches to analyzing the soft- worked. They wanted me to provide some kind of explana- ware increases the likelihood that serious problems will tion for why this code might fail. I didn’t have much to say. be detected and removed. Mistakes happen, so we test. Indeed, the purpose of testing I saw these differences in approach on my project. It is to observe whether the software actually works as its de- showed up in the test designs, where the developers signers imagine it will. But my developers sometimes be- sometimes didn’t see why certain tests were important. haved as if this attitude was a challenge to their honor. The tests, for example, may have been redundant from a Don’t presume that your testers need to have degrees 44 www.stqemagazine.com Software Testing & Quality Engineering Januar y/Februar y 2000
- 4. in computer science. Many excellent testers have training ed your defect report as ‘works-as-designed’?” There are in experimental sciences. Look for and appreciate this lots of good answers to this question. They might try to get background in your recruiting. more information from the developers about why they think it isn’t a defect. They might try to find a more dramatic in- stance of the defect that can’t be so easily dismissed. Or GOOD TESTERS they might contact customer support regarding their opin- Empirical ion. Here, the wrong answer is to not have an answer. What’s observed Testers need to demonstrate that they can continue to think Skeptics and work in the face of conflict. I want to see that they will be tenacious, and that they have some faith in their own GOOD DEVELOPERS perspective. Some people have strong technical skills, but Theoretical are very uncomfortable in these kinds of situations; these How it’s designed candidates rarely make good testers. Believers GOOD TESTERS Tolerate tedium Tolerating Tedium Comfortable with conflict Good testers realize that software testing can often be Report problems repetitive. They learn to accept this. Many developers hate repetition and are often very good at finding ways to GOOD DEVELOPERS automate it. After all, one major purpose of computers is Automate tedium to perform mental drudgery. Avoid conflict On my project, the developers initially expected to au- Understand problems tomate and improve much of the test process. I had a hard time discouraging this and getting them to focus on execut- ing tests and looking for bugs. It is a challenge to do repeti- tive work; but it’s important and often necessary. Good In Summary testers must be able to remain alert and attentive, even Appreciating differences is critical for productive teams. when applying the same test for the fourteenth time. Different approaches aid in finding solutions, and mutual respect dramatically improves group problem solving. Testers should not be judged according to developer crite- Living with Conflict ria. Empirical thinking is an asset rather than an inability to Good testers must not shy away from argument. It’s their think theoretically. A jack-of-all-trades should be appreciat- job to report bad news—and this is not always taken well. ed rather than criticized for being a master of none. Many Sometimes it’s hard to pinpoint the source of a bug. Is it a of the skills and attitudes that good testers demonstrate design bug, a coding bug, or maybe a documentation bug? contrast with what we often look for in developers. When Or perhaps the tester is just confused, and it’s not a real hiring testers, look for, and develop, these skills. Don’t just bug at all. Regardless, the tester’s job is to report the prob- settle for junior developers. Productive teams with both de- lem. Some testers may go overboard, thinking it’s their job velopers and testers need both skill sets. Just as developers to pass judgment on the developers or force them to do bet- have defined career paths, so should testers. To remain ter work. This is not helpful. But good testers need to be competitive in this industry, nurture both the skills of devel- the kind of people who are not afraid of other people’s re- opers and the different but equally important skills of actions to bad news. testers. STQE On the other hand, developers often need to avoid the emotional intensity that makes concentration difficult. It is Bret Pettichord is a test automation engineer at Tivoli not easy concentrating on complicated technical material Systems. He edits the Software Testing Hotlist hour after hour. I saw developers on my project take much (www.io.com/~wazmo/qa) and frequently speaks at soft- more time diagnosing problems than I would have. It is im- ware testing conferences. Bret can be reached at b.petti- portant to report problems as soon as they’ve been con- chord@computer.org. firmed, and good testers will report a problem once it is re- producible and narrowed down. I saw developers, however, move right into debugging, identifying the faulty code and the build that introduced it. Part of this is attributable to the fact that they had the skills and interest in debugging the code. But it also seemed as if they were reluctant to report that they had found a problem until they knew exactly what they had found. They wanted to avoid a situation where peo- ple started speculating as to who or what may have a caused a problem. When I interview potential testers, this is one of my fa- vorite questions: “What would you do if a developer reject- 45 Januar y/Februar y 2000 Software Testing & Quality Engineering www.stqemagazine.com