Abstract image of a woman sitting on books and studying on a laptop

The Testing Planet Issue 9

Rosie Sherry, profile picture
Rosie Sherry

This document describes an "exploratory test adventure", which applies concepts from storytelling games to software testing in order to make it more creative and collaborative. It discusses setting up exploratory test adventures with a storyteller, participants taking on roles, a fictional scene and mission. Interruptions, events and flexible paths can be used to simulate real-world complexity and keep the testing exploratory. Examples are given of exploratory test adventures run at a testing conference to collaboratively test a system in an engaging way.

TechnologyEntertainment & Humor
Related topics:
Software Testing Insights • Quality Assurance • Test Automation Tools
1 of 32
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
£5 By Lisa Crispin Humans have passed on his- tory and knowledge via story telling for millennia. Sharing our stories1 is a great way to share experiences that are likely to help others learn. Many of us have learned fam- ily lore or local legends sitting around a campfire or a dinner table. Those stories might have taught us something about how to be good members of society, or how to appreciate beauty. I myself learn best by example2 , so when I want to transfer ideas to others, I try to explain them via real exam- ples. Over the years that Janet Gregory3 and I have taught tutorials and classes, we real- ized that many people want to know: “Who else has had this problem that I’m having? How did they solve it? That solution might work for me.” When Janet and I started writing our Agile Testing book, we interviewed teams from around the globe, and asked them to tell us their stories. We found that many teams encountered similar obstacles, and were a bit sur- prised to see that many teams overcame these obstacles in similar ways. This felt like validation of our own experi- ence, and we included these stories as sidebars in our book. Judging from the feedback we’ve received from readers, many practitioners benefited Continued on page 5 By Martin Jansson and Greger Nolmark Testing is an activity that can be highly creative. In order to improve learning and testing we see improvisation as an important aspect. Improvisation can be done in many ways; the one we advocate is that of storytelling or role- playing. It is our belief that by taking aspects of storytelling into the world of collaborative testing we can increase our creativity. This is what we call an exploratory test adventure. Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing. We’re testing when we’re trying to find out about the extents and limitations of Continued on page 2 ALSO IN THE NEWS QA AND THE GREAT BOOKS To keep your quality as- surance chops fresh and to pick up new skills in... Continued on page 6 AN ENVIRONMENT FOR TESTING CREATIVITY Good, effective soft- ware testing requires methodical hard work... Continued on page 26 Telling OurStories FromTesttoDriventoDesign TESTING SMARTPHONE APPS For many people, the use of apps has become an integral part of ... Continued on page 20 Exploratory Test Adventure A Creative, Collaborative, Learning Experience November 2012 | www.thetestingplanet.com | No: 9 A fictional journey of a tester which might just sound very familiar - read all about it on page 22 Where will your testing lead you? THEEVILTESTER QUESTIONTIME More provocative advice for testers who don’t know what to do! Continued on page 24
2 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Aretesterscreative?Theactofcre- ationcouldbeseenasanactofgen- erosity;thetransformationofsome elementofyourpersonalityandintel- ligenceintoatangibleartifact.Creativity isvaluetosomebody.You,yourteam, yourproject,yourcommunity,theworld. The testing community is blessed with a great many creative personalities, writing books, running conferences and meetups, sharing their expertise by way of courses and seminars. Unleashing some intrinsic facet of their being into an unsus- pecting (but grateful) world. Some of their thoughts can be found in the fol- lowing pages, demonstrating that the creative spark can be ignited in many different ways and that the creative product can take many forms. Andlet’snotforgetthatThe TestingPlanetisthecreativeproductof themindsbelow.Wehopeyouenjoyit! Letter from the Editor Simon Knight K K Main story continued from page 1 the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before.” We begin with a definition of testing by Michael Bolton, quoted from the blog article Testing vs. Checking1 describing the extent of what testing could be. This definition highlights the aspects of the explorer or the adventurer, seeking new endeavours and finding new information about the world. We like the idea of being an adventurer who is out exploring searching for treasures, wearing the Indiana Jones kind of equipment, instead of the forgotten clerk behind the desk turning papers. One form of improvisation can be done through storytelling games. Here is one definition of story telling games: • A storytelling game is a game where two or more persons collaborate on telling a spontaneous story. Usually, each player takes care of one or more characters in the developing story. Some games in the tradition of role-playing games require one participant to take the roles of the various supporting characters, as well as introducing non-character forces (for example, a flood), but other systems dispense with this figure and distribute this function among all players. • Since this person usually sets the ground and setting for the story, he or she is often referred to as the “storyteller” (often contracted to “ST”) or “narrator”. Any number of other alternate forms may be used, many of which are variations on the term “gamemaster”; these variants are especially common in storytelling games derived from or similar to role-playing games. • In contrast to improv theatre, storytelling gamers describe the actions of their characters rather than acting them out, except during dialogue or, in some games, monologue. That said, live action versions exist, which are very much akin to theatre except in the crucial absence of a non-participating audience. This quote is taken from Wikipedia2 , which we see is fairly accurate in explaining what this is about. Notice the similarity with what we do when testing in a group, but where we as testers refer to it as telling a testing story. Something that is rarely seen in testing though is that the tester acts out in full the role in a specific situation. When testing like a specific user or trying to figure out what happens when encountering a specific event or obstacle, using the concept from storytelling games can help enhance the testing by letting it be more creative and if nothing else a lot of fun. When playing roleplaying or storytelling games you have a gamemaster or a storyteller. In testing we might instead have a coach, moderator, test lead or someone leading the exploratory test adventure. Just like storytelling, you rarely act on out in full when testing but instead simulate and act as if you were the user. What is an Exploratory Test Adventure? Participants As in traditional storytelling games we need to put together a number of participants with different roles. First we have the storyteller, in this case the test lead, coach or similar function. The storyteller is the one who leads the event and paints the main picture in which the participants should act in. Even though the participants should have some freedom to take the event to where they see fit the storyteller should always be there and guide them if they stray too far from the intended purpose. Secondly, we have the players, in this case the testers. As in collaborative testing the number of people participating can vary depending on situation. Thirdly we have other participants that for example might be observers and developers that can give and gain direct feedback and contribute with ideas on where to proceed in the adventure. Depending on what kind of team you have the participants and its observers will be differently composed. If you are in a cross-functional team, you should consider which roles the business analysts and the developers take on so that everyone step out of their comfort zone to increase the possibilities of the outcomes. Developers and business analysts could be observers of how the system or solution is actually used, how the participants play out a specific scenario. This could be an excellent way of getting early feedback on design and intended use. Scene This is basically the environment the participants are expected to act in. This could very well simply be the usual “test lab” but there might be some benefits to actually put the testers in a fictive situation. For example, the scene could be a pilot roll out with a certain department that is expected to use the application in a certain way. It is up to the Storyteller to create the scene and add background, actors and other details that are important to the participants. The more detailed the scene is, the better the creative experience will be for the participants. By bringing in events that might have happened in real life, the scene can become even more real. Replaying a scene that has happened to the storyteller can bring an interesting learning experience for all involved. Participants might do something completely different from what the storyteller did in the first place. Time In role playing and story telling games you usually know the time limit. It might be a short scene that you act in only meant to be played for a few hours. Then you might have what is called an adventure that could be played out in several occurrences. The last one is a campaign, which plays out during a longer time of many occurrences. We’ve only tried out the shorter versions when applying this to testing. It all depends on what you wish to learn and what you want to play out. The campaign timeline would fit when teaching students for a longer time, while the short hour version is applicable to events Continued on page 3 David Greenlees Thomas Harvey Simon Knight Rosie Sherry Michael Talks The team Hot Forum Post! Hiding bugs - http://bit.ly/hidingbugs Br ief His t orY OF Time a Fr enchEdition NEWSIN BRIEF Telerikrecentlyannounced the acquisitionof Fiddler, a WebDebugger,apopular toolcreatedtoinspectwebtrafficand ‘fiddle’withincomingoroutgoingdata.  http://bit.ly/telerikfiddler
3 Follow us at www.twitter.com/testingclub Continued from page 2 such as the test lab on conferences. Mission As with any game or adventure there usually is some goal that lures beyond the horizon. This gives a sense of motivation and pushes the players forward but it is quite common to not have a clear goal at the end of the road. In these cases it is the journey that is important. The end goal might very well change during the course of the adventure due to different events, for example the discovery of an area with unexpected critical issues. This will most likely shift the mind-set of the group. The ultimate goal might be obfuscated by lures that keep the participants astray or in confusion. Getting on the right path, as far as they know, can then be one of the sub-goals. There is an infinite number of ways this can be set up. Roles A storytelling game usually puts each player in a fictive character and this ties into the notion that testers could, or should, think in different ways. For testing we instead play a certain persona or role and act in their context. The available roles are closely connected to the scene created by the Storyteller. Something rarely seen in testing is the usage of character sheets when testing like a specific role or persona. A character sheet is a collection of background, skills, abilities and other useful information about the character. This could be made open to everyone or kept secret, these aspects and motifs of the personas will most likely lead to interesting discussions and reflections during debriefing. It can also contain information about how certain relations are between other roles or personas that might hinder or help the group. This might enable the tester to delve even deeper to understand the boundaries and limitations in which one character can act. Paths In an adventure the storyteller sometimes introduces an important intersection in the story. In some cases the paths look obvious, but it is up to the participants to try to choose their own paths. The storyteller should encourage creativity that may alter the path or break free in new directions that none had considered. In this way the whole group will be able to improvise in a situation that is unplanned and where it is uncertain where you end up. These unexpected twists and turns also gives the Storyteller a chance to hone their creative skills. Interrupts At any time during the exploratory test adventure the moderator may call for the attention of the participants, which pauses the scene. These interrupts may be pre-planned or invented along the way. One reason to do this could be to get the testers back on track if they deviate from set goals, another may be to just stop and do a quick reflection on progress so far. While yet another could be to zoom out and move an imaginary camera to another context to show something happening that will affect the participants, as a flare of bringing the story onward or letting them know something wicked is coming their way. Reporting In some role playing games the participants keep a journal on what happens, who said what and new plots that appear. Over time there is a need to keep track on the many events. This is quite similar to how sessions or threads are documented in SBTM or TBTM. The most interesting result is when many parties tell their story in these journals, making it obvious that everyone experience different things. Events The moderator may have certain events pre-planned that are designed to trigger new ways of thinking within the group. An example of this could be that the moderator describes what is going on during a meeting elsewhere between Project Leader and Manager that most likely will influence how the group acts. Another example is to have an invited guest come along with a different agenda to disturb the groups testing session. There can always be ulterior plots and events that are happening that might enhance the adventure or set obstacles in the way of the participants. Examples of Exploratory Test Adventures Let’s Test Lab 2012 In the Test lab there were three tables that focused on one system each. Each table also was a team led by one team lead. The first objective given out by Martin and James was to do collaborative test planning in the teams finishing with a debrief on what they had come up with. At this stage there were no interruptions. The next part was actually Follow us on Twitter! @testingclub @testninjas @testingfeeds @stcjobs Br ief His t orY OF Time a Fr enchEdition Martin Jansson, owner and consultant at Testverkstaden Sverige AB, started his career as tester 1996. He has tried many professions in product development, but his heart and soul belongs in testing. Mar- tin is one of the founders of www.thetesteye.com which has grown into one of the greatest Swedish blogs on software testing. Martin is a frequent runner of the testlabs at various test conferences. The last years Martin has assisted clients in evolving their organizations from a traditional one into an agile, where he focuses on what aspects of testing that works in an agile environment. You can reach him on Twit- ter @martin_jansson or on martin.jansson@testverkstaden.se AUTHOR PROFILE - Martin Jansson investigating what they had planned. During this session Martin interrupted and introduced a scene in which the test lead appeared on his way to a release meeting. He had just a few minutes before he needed to attend, so each team needed to quickly create a report on the most important issues and present them to Martin. The diversity in how the teams presented was very interesting. Some of the team members and some of the listeners thought the learning experience was great. The Test Lab was filling with over 50 people. The original plan was in the trunk. Each table consisted of at least 5 testers, and then there were many who stayed in the background listening and discussing what was going on. The scene was set by stating that they were testing teams, Martin and James took on the roles of project manager, line manager, test leads and CEO. Martin and James had from the beginning stated that they would go around the tables and say different things, thus contradicting each other. Present were also Ilari Henrik Aegerter and Anne-Marie Charrett who did coaching. There was great confusion in the teams since they were split between the objectives given out by Martin and James, while they at the same time wanted to work on their own agendas and while Ilari and Anne-Marie were trying to get their attention in coaching. All of us were improvising; the scene (what we thought it was) was changing and evolving all the time. During the whole session there were a few interruptions that presented the scene and its actors with new information, on which they could act on. During one of these interrupts Martin, as a project manager, asked why the teams were using the coaches, they had so much testing to do and so little time! This changed the team’s focus and removed some of the confusion. The experience was very different for the participants, but it seemed many enjoyed it and some learned more than others. (See3 for further details) Continued on page 4 Character Sheet: Local IT Support Background: Working at Customer IT Dept. Has been assigned to Acceptance Test Team due to high experience with previous releases of similar products. Focus/Interest: Tech - Install, Database, Performance Secret Plan: This release will most likely threaten the need for Local IT support. This due to plans to outsource support in connection with this launch. You are looking to discredit the product in any way possible.
4 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Hot Forum Post! Exploratory Regression Setup? http://bit.ly/exploratoryregression Continued from page 3 Other disciplines that could inspire us If we look into the world of organisation theory, there are movements that also investigate ways to become more creative. Here are a few quotes from the article Improving Case Discussion With an Improv Mind-Set by Andy Aylesworth4 that align well with our thinking of improvisation in testing. “To understand better and explain the phenomenon of organizational improvisation, scholars have turned to improvisational arts, specifically theatre and jazz, where improvisation is the norm, rather than the exception.” “Improvisational theatre has been invoked to better understand business innovation, management, collaborative technology, and team performance.” “... Employees to attend improvisational theatre classes, believing that such classes will lead to better, more productive, and creative employees.” By improvising they can explore a concept and try new ideas to better understand and perhaps learn something new. The concept comes from that of a Case study, which is very like that of running a scenario or setting up a scene to act in. The author thinks the use of improvisation inspired from the theatre would increase creativity when using the Case study technique. He also believes that participants, who usually are either rowdy or shy, might step out of their usual role and act in a different way. Participants in an exploratory test adventure would probably have a similar experience. When you take on a role that you might not otherwise have experience from, you need to consider new things and explore the boundaries in which you can act. In this situation your creativeness is key. The book Body and Language: Intercultural Learning Through Drama edited by Gerd Bäuer5 highlights several aspects regarding improvisation that aligns well with our ideas on exploratory testing adventures. Here is a section that is especially interesting from that book: “At least until computers can recognize Br ief His t orY OF Time a Fr enchEdition Greger Nolmark, Senior Test Consultant at Adecco IT-Konsult, started working with test in 1999. During the years he has had several different positions in the areas of test and support. His primary focus has been on testing always keeping an eye on the end user perception. Dur- ing recent years he has nurtured an interest in how new and different test techniques can help teams and testers to test better and to show there are more to testing than being certified. You can reach him on greger.nolmark@adecco.se or Twitter @GregerNolmark. AUTHOR PROFILE - Greger Nolmark to the exploratory test adventures is that of Experiential Learning. Gerald M. Weinberg has recently written a book about this, but you can also find lots of interesting blogs and such on the topic as well. David A. Kolb has researched a lot on this topic. “David A. Kolb (with Roger Fry) created his famous model out of four elements: concrete experience, observation and reflection, the formation of abstract concepts and testing in new situations. He represented these in the famous experiential learning circle that involves (1) concrete experience followed by (2) observation and experience followed by (3) forming abstract concepts followed by (4) testing in new situations (after Kurt Lewin). It is a model that appears time and again.” A quote from an article by Smith, M. K. (2001) on ‘David A. Kolb on experiential learning’6 . Experiential learning goes hand in hand with that of an exploratory test adventure. By setting up a scene in which we act, with events and encounters that simulate our everyday experiences we will learn new things and enable us to be creative in its context. Conclusion To use improvisation to be more creative and to enhance learning is not a new concept, still it is not often seen as part of people’s repertoire. Part of the reason is probably that many testers for different reasons work alone, instead of collaborating. Exploratory test adventures prepare the testers for the unknown by forcing them to go together through various simulated experiences where they need to improvise, which in turn nurtures creativity. We can even say that the foundation of a storytelling game is improvisation and creativity. This is why we promote that you, as a test lead, team lead or team member, step up and take the role of the storyteller. Play out a scene in which your team members can play an exploratory test adventure! □ REFERENCES 1. Bolton, M. (2009). ‘Testing vs. Checking’. Retrieved 2012-09-14 from http://www. developsense.com/blog/2009/08/testing-vs- checking/ 2. Wikipedia. ‘Storytelling games’. Retrieved 2012-09-14 from http://en.wikipedia.org/ wiki/Storytelling_game 3. Jansson, M. (2012). ‘A Let’s Test Lab Story’. Retrieved 2012-09-14 from http://thetest- eye.com/blog/2012/05/a-lets-testlab-story/ 4. Aylesworth, A. (2008). ‘Improving Case Dis- cussion With an Improv Mind-Set’. Retrieved 2012-09-14 from http://jmd.sagepub.com/ content/30/2/106.full.pdf 5. Bräuer, G. (2002). ‘Body and Language: Inter- cultural Learning Through Drama’. Retrieved 2012-09-14 from http://www.google.se/book s?id=xa2xD9pHonIC&printsec=frontcover&hl =sv#v=onepage&q&f=false 6. Smith, M. K. (2001). ‘David A. Kolb on experi- ential learning’, the encyclopaedia of informal education. Retrieved 2012-09-14 from http:// www.infed.org/b-explrn.htm and represent aural human speech a lot better than they can now and can be programmed to respond spontaneously to speech (which I, for one, don’t believe will ever happen), one cannot learn to creatively engage in a conversation in a language unless one has real human beings to interact with. Audiotapes and computer language programs can help one learn certain common exchanges or routine phrases, but to learn how to improvise new utterances one has not yet heard, at least one other speaker of the target language is needed. This is why informal improvisation drama activities are so powerful in the foreign-language classroom. To participate in an improvisation, one needs to use the body not only to produce appropriate language but also to express emotion and ideas through gesture, posture, and facial expression. Because the scene in a drama is an imaginary one, the participant is free to exaggerate or assume a persona that frees him or her to experiment with a wider range of language that ordinary exchanges might evoke. Improvisational drama is effective because of the repeated pressure it puts on participants to respond. It is not enough for students to hear the target language spoken; they need to talk themselves.” Notice the implication that by stepping out of your everyday situation into a context where you are not familiar and where your usual boundaries lie, you instead visit a world that can be explored in many different ways. All participants are part of exploring this new world and act in it according to their own set of beliefs and objectives. We see that the complications in learning new languages and the experimentation using improvisation align well with the complications of creating an environment that enables creative testing. Cem Kaner has expressed the complexity of teaching testing and many others have argued about the lack of actual knowledge gain by merely studying a curriculum such as that from the ISTQB. We see that there are similarities in teaching to express yourself in a new language with that of performing creative, context-driven testing. As you might figure out, improvisation and following a script does not go hand-in-hand. Another concept that is very applicable Character Sheet: HR Department Background: Working at HR department with recruitment. Has been assigned to the Ac- ceptance Test Team due to experience with similar systems. Focus/Interest: UI – Layout, Spelling, Error handling Secret Plan: Quick and dirty testing today. Real vs Barcelona on the telly tonight.
5 Follow us at www.twitter.com/testingclub REFERENCES 1. http://storycorps.org 2. http://exampler.com 3. http://janetgregory.ca Second story continued from page 1 from these experiences of other teams. Testers want to hear the real-life experiences of other testers. It’s simply another form of passing on wisdom via story telling. When I meet a new teammate or colleague, I want to hear their story. I might ask, “How did you come to be here?” Always, we discover we have several shared experiences. If we’re working together to build a high-quality software project, this helps us build a foundation to collaborate effectively. I like to hear stories from people with dif- ferent points of view. Each person who participated in an event tells a different story about it, depend- ing on their context, their perspective, and their experiences. We need all these different viewpoints in software projects, too. Each viewpoint has equal value. The diversity of viewpoints fleshes out the “big picture”. When your team must deliver a new fea- ture, ask a customer to tell her story with respect to that feature. Ask for examples of desired and unde- sired behaviour. Don’t constrain them, let them cre- ate! In my experience, many of us learn well from concise examples. We can use the same technique to specify how a software feature should behave. We can use all the different perspectives to help us think of more ways to explore and test the feature. Storytelling is a creative process. Software development (including testing) is also a creative process. Combine the two, and you have powerful ways to understand what your customers desire, and to make sure those desires are fulfilled. □ AUTHORPROFILE-LISACRISPIN LisaCrispinistheco-author,withJanet Gregory,ofAgileTesting:APracticalGuide forTestersandAgileTeams(Addison- Wesley,2009),co-authorwithTipHouseof ExtremeTesting(Addison-Wesley,2002), andacontributortoExperiencesofTestAu- tomationbyDorothyGrahamandMarkFew- ster(Addison-Wesley,2011),DevOpsfor DevelopersbyMichaelHuetterman(Apress 2012),andBeautifulTesting(O’Reilly,2009). Sheenjoyssharingherexperiencesviawrit- ing,presenting,teachingandparticipatingin agiletestingcommunitiesaroundtheworld. Shealsoenjoyslearningbetterwaystodeliv- erqualitysoftwareworkingasatesteronthe PivotalTrackerteam.FormoreaboutLisa’s work,visitwww.lisacrispin.com.@lisacrispin onTwitter,entaggle.com/lisacrispin Stories The power of story-telling Tell me your story How did you come here? Stories are examples We learn from examples People value "real-life" experiences Ask a customer to tell his/her story Helps understanding Can be used as example & tests Desired behaviour Undesired behaviour Different participants in an event tell a different story Each viewpoint is valid Diverse viewpoints "flesh out" the "big picture" NEW BOOKS Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing By Elisabeth Hendrickson http://bit.ly/ehexploreit  Tap Into Mobile Application Testing By Jonathan Kohl https://leanpub.com/ testmobileapps Cucumber & Cheese - A Testers Workshop By Jeff Morgan https://leanpub.com/cu- cumber_and_cheese ATDD by Example: A Practical Guide to Acceptance Test- driven Development By Markus Gärtner http://amzn.to/WD5IWI Br ief His t orY OF Time a Fr enchEdition Figure1.StoriesMindmap Training Course: Coaching Testers - http://bit.ly/coachingtestersmot Br ief His t orY OF Time a Fr enchEdition
6 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Training Course: Techniques for Agile Manual Testers - http://bit.ly/agilemanualtesters QA and the Great Books Br ief His t orY OF Time a Fr enchEdition Brian J. Noggle has worked in quality assurance and technical writing for fourteen years in a variety of industries and situations. He currently works as a freelance software-testing con- sultant through his own company, Jeracor Group LLC, and specializes in short-term projects with limited budget. Noggle’s novel, John Donnelly’s Gold, is a comic romp in the world of IT. Although it be classical literature yet, he thinks you should read it. AUTHOR PROFILE - BRIAN J. NOGGLE By Brian J. Noggle To keep your quality assurance chops fresh and to pick up new skills in the technology field, almost every QA professional reads in the field. This includes blogs, Web sites, magazines, and white papers or longer by luminaries in the field such as James Whittaker, James Bach, or Cem Kaner. These sources can keep you up to date on the latest trends and thoughts in established quality practices and methodologies, but ultimately will only lead you to thoughts other people have thunk first. While you can successfully apply these quality precepts to your situation and can combine them with your experience, if you really want to become a thought leader and to make greater cognitive leaps, you’ll need to provide material for that great creative synthesizer between your ears. When you read books from outside the field, you imbibe ideas that you can integrate with your industry knowledge to invent new techniques and new ideas. While the impact might not be immediate, all information you take in becomes fuel for the creative fusion process. Years after you’ve read something, its lesson can come unbidden when you’re faced with a new problem or the same old problem in new circumstances. Or maybe the colour of the sky will remind you of the scene from a novel that will lead you to apply a situation from that book to your current project with significant results. If you’re game and want to challenge yourself (only a little bit, since most classical literature is actually pretty approachable if it does not intimidate you), I have some suggestions for a reading list. Books To Motivate In the quality assurance world, the whole de facto development process seems aligned against us. Every deadline seems achievable to management as long as the team can take those extra days out of the test cycle. The defects you’ve found, analysed, and advocated (the correction of, not the defects themselves) get ignored, put off into the never-arriving future, or otherwise removed without remediation. You return to your cubicle after the meeting and ask yourself, “Why am I doing this?” Sometimes you ask that when you get to your desk in the morning. Classical literature is rife with heroes who encounter difficulties in their day-to-day and carry on. The Bhagavad-Gita tells of Prince Arjuna, a member of a royal house who is off to war with some other royal houses, some of which are family. Arjuna doesn’t want to go to battle. His attendant Krishna explains to him how the world works and explains to Arjuna that it’s his duty and role in life, so Arjuna eventually does. If you’re looking for a little more Western perspective, pick up a piece of classic noir fiction by one of the masters of the 1930s and 1940s, whether it’s Raymond Chandler or Dashiell Hammett. Throughout their books, world-weary private investigators try to bring a little justice to the world by solving their small cases. Frankly, these books are closer to software quality assurance than they should be, with our need to make the world a better place through isolated victories in education or process improvements. Sometimes, though, a sap to the back of the head doesn’t sound too bad compared to another lessons learned meeting. Books About Time Management and Self-Improvement With hundreds of deadlines and millions of tasks assigned to QA every cycle or sprint, sometimes you can get a little overwhelmed and not even know where to begin to get it all done nor how to improve your skills and techniques outside those other demands. Several works of classical literature spend some time on those very topics. The Autobiography of Benjamin Franklin, written by Benjamin Franklin, tells you what a wonderful fellow Franklin was. Although self- promoting, Benjamin Franklin, the son of a candle maker, took a printer’s apprenticeship and parlayed that experience into founding a magazine, entering the political realm, and serving in many early American government positions before and after the Revolution. To accomplish this, he worked very hard and kept a rigid schedule of self-improvement. It’s harder in the 21st century to keep on task and on a tight schedule with the distractions of the Internet and mobile devices, but Franklin’s discipline can offer some guidance and inspiration. You can find a fictional flip side to Franklin’s discipline in The Great Gatsby by F. Scott Fitzgerald. The title character also rises from humble beginnings to a position of opulence through rigid self-discipline. As a Lost Generation novel, though, the story does not end as well for the characters as Benjamin Franklin’s does for him, but The Great Gatsby does feature more cloche hats. Books About Adapting In the hurly-burly world of software development, where the battle’s lost and won daily, the world shifts daily in subtle or dramatic ways. Methodologies change, underlying technologies change, and the participants keep getting younger. In this world, you have to adapt quickly to changes, and classic literature can help you in two ways. First, because literature can extend from Chaucer to the 21st century, you have to quickly adapt to different languages and different eras Continued on page 7
7 Follow us at www.twitter.com/testingclub Who needs questions? Lets just test! http://buff.ly/RxcOqR Br ief His t orY OF Time a Fr enchEdition Continued from page 6 just to read the books. A Jane Austen novel circa 1813, a Charles Dickens novel circa 1835, and a Thomas Hardy novel circa 1886 differ from each other in setting and in language, and they’re all quite different from the aforementioned The Great Gatsby. By varying your reading, you’ll train yourself to adjust your in-reading understanding to the work itself. This habit will transfer as you change between development projects and between problem solving for different discrete industries with their own concerns and argots. If you shift your mental gears between the country estates, dismal factories, and the wilds of India without grinding them, you should easily adjust between a client or project in the Internet retail space and a client or project writing a mobile application for courier tracking and delivery. Secondly, you can watch and perhaps learn from characters that adapt quickly. If you take Rudyard Kipling’s Kim, you’ll learn from a young man who moves between Buddhist, Moslem, Indian, and British worlds as he participates in the Great Game. As a bonus, if you grab a collection of Rudyard Kipling’s works that includes Kim with Kipling’s short fiction, you’ll develop definite agility in switching contexts, as Kipling’s characters range from Americans in England, English soldiers in India, English officers in India, naval officers at sea, a mongoose, and more. Not-So-Great Books Of course, the springboards for your creative vaults do not have to come from the Great Books. You will find ideas and inspiration in other books, too, if you’re open and attentive. You might find solace and camaraderie in the hardboiled noir books or in the second wave pulp paperbacks of the 1960s through 1980s; I certainly fancy myself a Mack Bolan of testers. Dan Brown’s puzzle-and- conspiracy filled tomes might give insight into how to approach an unfamiliar interface with an eye to reading the messages and alternative workflows that others do not see. Maybe a comic IT heist novel will satisfy your revenge fantasies with a side order of physical system analysis and problem solving. You can also glean ideas from nonfiction books, particularly in other specialties. Perhaps a book about real estate sales can illustrate interpersonal relations in a new way and help you to communicate more effectively. Old technology or forgotten management style books might include ideas that you can tweak for the present day. Biographies of military leaders provide management ideas and lessons, only some of which involve harrying the north. Regardless of what you choose to read outside the field, you’ll make greater leaps of creativity when you add unrelated content to your mind. You’ll have more material from which to draw to try new things and to see things differently from your peers. At the very least, you’ll learn new quotes to sound classically educated and Latin words to brandish like gladii. As a child I devoured the Isaac Asimov “I, Robot” series - they were wonderful stories. The world of the future had robots bound by 3 Laws of Robotics. However despite this, these machines would behave in odd and occasionally dangerous ways. Invariably the cause of this under investigation would be some unperceived complexity in a scenario or situation, which was overlooked. I think particularly the tales of Powell and Donovan spoke to me as “hero testers”. I think these stories taught me a lot how to investigate and challenge “bugs” (or rampaging robots), especially when others say that a problem in their area of code is “inconceivable” (also see The Princess Bride for use of that word). “What Do You Care What Other People Think?” by Richard Feynman is an inspiring and life-changing read. He was a physicist on the Manhattan Project, and the book talks a lot about his scientific outlook on life. A good portion of it involves his work on the investigation into the Space Shuttle Challenger disaster, and how he encountered a culture at NASA which believed too much of it’s own hype. Some notable quotes from this book include “When playing Russian roulette the fact that the first shot got off safely is little comfort for the next” and “reality must take precedence over public relations, for nature cannot be fooled”. IntheTestingPlaneteditorialteamwewerereallyimpressedwithBrian’sarticleasitechoedourownexperience.Sothe questionwentaroundourvirtualoffice“whatnon-testingbookshaveinspiredyouinyourtesting”.Hereareouranswers... Br ief His t orY OF Time a Fr enchEdition How to Use Your Eyes by James Elkins - It’s hard to capture in a few paragraphs what this book has offered my testing, as every time I reflect on it, I MIKE TALKS DAVID GREENLEES think of areas that it can be extended to. One notable mention would be the prompt throughout the book to view beyond what you’re seeing. I like to refer to it as the ‘extension of sight’. Without giving too much away, the first section of the book focuses on postage stamps. Yes, there is time spent describing the actual picture on the stamp, which is fascinating, however the background story that the picture tells is the most amazing part. This book constantly reminds you to think about that story. When I first began reading it I said to myself, “Oh no, not postage stamps... how boring”. It ended up being very interesting and very educational. Since reading this book I have remembered to look beyond what’s directly in front of me. This has helped me in many aspects, however most noteworthy would be gaining an understanding of why certain decisions are made. Instead of looking at a decision in isolation, I now seek that background story. It’s a powerful thing to find out what drives us as human beings, and all we have to do is employ an ‘extension of sight’. The Ender Series by Orson Scott Card - Not one book but a series, the story of Ender (or Andrew) Wiggin by Orson Scott Card has had a big influence over my thinking since I read it about six years ago now. It really is an outstanding piece of science fiction, but in addition to this it introduced me to the subject of anthropology. Being primarily the study of human behaviour, the author applies this science to the observation of alien races resulting in some quite fascinating story arcs. Applying this thinking to the dynamics and behaviours of project teams has been and will no doubt continue to be a fruitful source of valuable information. Wiggin goes on from the seminal Enders Game to become the first Speaker For The Dead, which entails studying the life and times of a deceased being and telling their warts and all story by way of a public speech. To quote Wikipedia on the matter - “This speech is not given in order to persuade the audience to condemn or forgive the deceased, but rather a way to understand the person as a whole, including any flaws or misdeeds.” The quote could almost be a testing mission statement - a mandate not to judge, but to provide a common understanding of the product as a whole, including any flaws or misdeeds. I like it! Culture Shock: A Handbook For 21st Century Business by Will McInnes - I’ve long struggled with how traditional businesses are run. The more I experienced working in ‘them’ the more I became curious of understanding what it takes to create a business. As I’ve built up the Software Testing Club I’ve always known it would be built lean, on a budget, but with high integrity in mind. Culture Shock is a great book that covers what many forward thinking companies are doing to approach business in a different way. Some things these companies do seem crazy, but it doesn’t stop them being successful. Lean StartUp by Eric Ries - This follows on from theme business theme behind Culture Shock. I love business and am always coming up with new ideas of what we could be doing. Some are small additions to what I’m already doing, others are inter-related... and of course there are the pipe dreams. What it has done is allow me to focus on what I’m doing, taking things one step at a time and evaluating things as I go. I haven’t been a strict ‘lean startup’ but the philosophy is very much in my heart. □ ROSIE SHERRY SIMON KNIGHT
8 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Are You in QA? No. Am I a Software Tester? Yes! By Michael Larsen The above statement may seem a little flip, but I have this question asked of me quite frequently. Somehow I will be involved in a discussion and something will come up about testing, and when I talk about it, a natural question to be asked is “oh, so you must be in Quality Assurance!” For years, I answered “yes”, but I don’t answer that way any longer. Now, when people comment that I must be in QA, I answer, “Am I a software tester? Yes!” For many of the people I talk to, the comment doesn’t just hang in the air; they notice the difference, and often they ask me about it? What do I have against QA? Why do I avoid saying those words, and why do I substitute the words “software testing” instead? The roots of this come from my further understanding of the dynamics and the environment(s) that make up software, how it is created, and the environment in which it is built and where it runs. The term Quality Assurance stems from the idea that there should be quality standards. software exists in an unusual state that is never 100% duplicable. If we were to take two computers, loaded with the same operating systems, and with as close as possible the same hardware, and we ran those two systems, with as close to identical configurations as possible, we still could not, with certainty, say that the machines would behave identically. We could guarantee that the metal parts were stamped out to make the case with a high amount of precision, but the sustaining of the electronic states of an operating system will be different on each computer. The vagaries and differences in the speed of ram, the speed of the hard disk, the length of the cables used inside the chassis, the power load at that given moment, and any programs installed and activated on the system will make for a different experience. This is a key reason why I have come to view software “quality assurance” as a misnomer. Metaphorically, installing and testing software is less like building a house on a concrete foundation, but more like building a houseboat that floats on a lake (and to extend the metaphor, building it on the lake instead of on solid ground). The reasons run even deeper than this however. With Quality Assurance, there is both an expectation and an understanding that we can do something about the issues we find. That’s what Assurance means. I assure that this product has a level of quality. For most of us that work in this space, that is not likely, nor is it really even feasible. To assure software quality, we would need to have the ability to make changes to the system, to “retool the line” so to speak, so that we can make sure that the system is doing exactly what it was Continued on page 9 This also stems from the factories in which items are made. Whether they are made of wood, plastic or metal, factories have a vested interest in their parts being designed to have few defects or broken pieces. The idea of quality assurance grew out of the factories and assembly lines looking to guarantee, as much as possible, the parts that were made not only fit the correct measurements and specifications, but that they also fit into the other parts to assemble a finished product. If we are talking about a drill press, and the ability to make for equally distributed holes into a piece of sheet metal, having the ability to guarantee that process works consistently is realistic. It’s a very stable process, and something that can be assured (or as close to a guarantee as is possible). The metaphor of a factory and an assembly line feels natural when we talk about software. Functions plug together, modules can be connected to make larger programs, and numerous “moving parts” can be “assembled” to make a finished product. The similarity, however, ends here. Unlike a drill press or a metal stamping machine, MichaelLarsenisaseniortesterwithSideReel.com,awhollyownedsubsidiaryofRoviCorpora- tion,currentlyworkingattheRovisiteinSanFrancisco,California.Overthepasttwodecades, hehasbeeninvolvedinsoftwaretestingforproductsrangingfromnetworkingequipmentto capacitancetouchdevicestoInternetapplications.MichaelservesasaDirectorfortheAs- sociationforSoftwareTesting(AST)andistheChairoftheEducationSpecialInterestGroup. HeactivelyteachessoftwaretestingthroughtheBlackBoxSoftwareTestingseriesofclasses offeredbyAST.HeisablackbeltintheMiagi-doSchoolofSoftwareTesting,istheco-founder andprimaryfacilitatorforWeekendTestingAmericas,andistheseniorproducer(andfrequent commentator)forSoftwareTestProfessional’s“ThisWeekinSoftwareTesting”podcast. AUTHOR PROFILE - MICHAEL LARSEN WWW.MINISTRYOFTESTING.COM CO-CREATING SMARTER TESTERS EDUCATION • COLLABORATION EVENTS
9 Follow us at www.twitter.com/testingclub We’ve created an ultimate resource on SBTM - http://buff.ly/QlZkO0 Br ief His t orY OF Time a Fr enchEdition Continued from page 8 specified to do. In the software-testing sphere, at least in the environments that I work in, we don’t have that control or power. We have to have a programmer make those changes. Thus, the ability to assure quality is not in our hands. The biggest reason, though, has nothing to do with these situations I’ve described. The reason I dislike using the term “Quality Assurance” the most is the false expectation it gives. It tells people that we can assure quality, in a way that says we will prevent any defects from getting to the public. While yes, that is ultimately what we want to do, the fact is that it is impossible to test completely, to examine every conceivable situation, and cover every eventuality to assure that there are no defects. If a defect does get out, we have set ourselves up for blame. We are Quality Assurance, our job is to assure the quality of the product, and therefore any bug found in the fields is our fault. Nonsense. While it is true that I, as a software tester, did not find that particular issue, neither did the programmer who wrote the module, or the product owner who accepted the module. We’re all on the same team. Setting ourselves up to be the judge, jury and executioner will merely leave us to be blamed if we miss something. These are of course all semantic arguments. The fact is, Quality Assurance is a long established term, and for the most part it is synonymous with being a tester. Still, I prefer using the words “software tester” for a variety of reasons. First, it accurately effects what we do. We test the software. Not an idealized piece of software on some mythical perfect machine, but software that flows and sways like that houseboat on a lake. We look at a product and we explore it. We discover with the resources at our disposal how the software behaves, and based on that behaviour, we explore in different places. Jon Bach has a metaphor that I personally love1 , in essence saying that instead of trying to present ourselves as the “bug shield’ or the “last tackle on the field” we should use something closer to who and what we really are. We are storytellers. We are journalists. Our goal is to explore, study and learn about a product, from as many different vantage points as we can, and then, based on those vantage points, explain what we see. Tell as complete a story as possible. Show the software team and product owners the who, what, where, when and why of the software. We are providing the development and product team, and the customers, with information about the fitness of the product. Ultimately, the product team and the programmers will determine what they will do with that information. Often it will be to take the problems that we have found and provide fixes for them. Sometimes, it will be to leave the issues as is. Each time I have this conversation, I have the chance to change someone’s mind on how they see software development and what we as testers really do. Often, I find that I cannot change opinions; the long-term use of the QA term is too strong and ingrained. But sometimes I find someone who gets what I’m saying, and they start to view software testing differently. I’ve even heard a number of software testers likewise answer in kind when they are asked if they “work in QA?” It’s going to take time to change perceptions, but if we all work together and remind people what it is we really do, we might help others understand that the term Quality Assurance belongs in the factory, while software testing belongs with the code we work with every day. □ REFERENCES 1. http://agile2010.agilealliance.org/files/Tell- ing%20Your%20Exploratory%20Story%20 Agile2010.pdf Visit www.qawizard.com to learn more and download your free 30-day trial today. Learn More QA Wizard: Intelligent Tools for Testers Automate your web, Windows, and Java testing, and gauge the real-world performance of web sites with QA Wizard Pro. • Create adaptable scripts that require less maintenance with an object-based recording engine and global control repository • Test the latest technologies including HTML 5, Java, .NET, Qt, AJAX, and more • Simulate hundreds or even thousands of users to measure the real-world performance of your web site With Resource Thief you don't need to maintain virtual machines, or create crazy configurations in the lab to test application behavior in stressed environments. • Limit disk, memory, and network access to specific applications • Run existing, or new test cases against the application under test • Use your existing diagnostic, testing, and development tools during stress tests Resource Thief Stress Testing QA Wizard Pro Automated Functional & Load Testing
10 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Next Rapid Software Testing course is in March with James Bach in Brighton! http://buff.ly/Qd3Py9 Br ief His t orY OF Time a Fr enchEdition A Creative Tester Asking for Trouble By Jeffrey H. Lucas I always used to get into trouble. It always started the same: I would be given a straightforward task of running a pre-defined, established test script written resulting from an analysis of the requirements that was conducted at the very beginning of the project. I would start the procedure as usual, verifying the test script appeared to address the requirement, and begin running it. Testing is not about following a script, but applying your imagination and creativity to the testing of the product. However, if applied in a static fashion, scripted procedures tend to suppress this creativity. To illustrate this, I provide two test sessions involving a fictional inventory tracking system for a merchandise distributor (“Foobar Mercantile Corporation”). The purpose of this imaginary application is to provide greater visibility to the product inventory in transit from the warehouses to the distribution locations. Some mockups are provided to help follow the testing. Session 1 – Following the script The first session involves an 18-step pass/fail “scripted” procedure. The stated purpose of this procedure is to demonstrate the requirement for a system administrator to provide reporting capabilities to groups. The procedure has been verified (i.e. run) multiple times and is expected to take about two minutes to complete as a part of the acceptance test process for this delivery. The procedure demonstrates the requirement by assigning the reporting permission to the default Domain Administrator group (who administer geographic/physical locations within the system). In this procedure, I represent my thought process using imaginary users I create in my mind to visualize the application in use. Compare the tangential, sometimes chaotic, thought lines to the simple flow of the script. ME: Are we set to run the procedure? CO-TESTER: Ok, let’s do this and then we can take lunch. I will call out the steps and you execute. “Step 1. Login as a System Administrator and … “) (Keyboard-restricted User) ... I can’t navigate out of the login area (Ref. 2) using the Tab key ... I have to use Ctrl+Tab … very difficult … I want to access the company link at the bottom... (Company President) ... Why is our cor- porate logo on the company link (Ref. 3) so blurry looking? … Very bad for our corporate image... (Keyboard-restricted User) ... I can access the page … navigating back, but the focus appears to move to the top of the browser with no indication of focus (Ref. 1)… this is getting -very- difficult to use … Me: I’ll make a note of those. Co-tester: “... add the ‘Reporting’ permis- sion to the Domain Administrator group.”) (SystemAdministrator) ... give me a break … I’ve been doing system administra- tion for 9 years and you want me to give that permission all of the Domain Admins? ... Me: So what are you saying? This function is a basic, established part of most applica- tions. A best practice that was identified as a necessary product enhancement early in the process. I read it right here. (System Administrator) ... if you knew the Domain Admins like I do, you wouldn’t be so... (DomainAdministrator)...Iresentthat... Me:Wait...soyoumustbeaDomainAdmin.... (DomainAdministrator) ... no I got moved to section management in trans- portation, but they still have me overseeing some of the warehousing administration... Me:Soyouhavemixedroles!Thatmaymake addingthe“Reporting”permissiontotheDo- mainAdministratorgroupabitofaproblem. (Ex-DomainAdministrator) ... Yup, I agree that it shouldn’t be applied unthink- ingly. I do have a need for it, so perhaps a limited application based on job description and not just titles... (AdministrativeAssistantw/de- mandingboss) ... Wait a minute; don’t forget about me ... I also need access to support planning... Me: So you think the current group permis- sion implementation may be too limiting and should be rethought? (Administrative Assistant w/ de- manding boss) ... yeah, maybe a special group assignment based on identified need rather than administrative title or formal job description... (Ex-Domain Administrator) ... that’s doable... (System Administrator) ... yeah, I could go along with that... Continued on page 11 Figure1:LoginPage
11 Follow us at www.twitter.com/testingclub iOS Testing MindMap - http://buff.ly/QRSh2J Br ief His t orY OF Time a Fr enchEdition Continued from page 10 Co-tester: “Step 2. The Domain Admin- istrator receives an email notifying them of the permission change with a link to the application login page.”) (SystemAdministrator) ... Whoa! That’s just asking for a spear phishing or whaling attack... (BlackHatattacker) ... Heh heh... Co-tester: OK, verified. Next: “Step 3. Login as a Domain Administrator, open the permission panel, and verify the Reporting permission state has changed from green to red.” (Colorblinduser) ... Now, exactly how am I supposed to tell that?!? (Ref. 4)... Me: Ok, hold on. I think there may be some serious issues with this user flow. I’m see- ing a lot of areas that we need to stop and explore for a bit. Co-tester: Again!?! Now why do you think that? You know they expect us to have this test suite completed before we leave today. Me: Yeah, I know. Let’s just say a little voice told me. This is just a sample of how restrictive test processes can inhibit the creative imagination of a tester. Many testers with a wealth of experience in testing have experience similar sessions in the past. The key is to incorporate the imaginative, inquisitive mind into the testing. Session 2 – Putting your imagination, not your script behind the wheel To leverage the power of a tester’s imagination, you have to give a vent to the thoughts – thus giving an actual voice to the imaginary users. The second example involves a test session with a stated charter to explore the requirement for a system administrator to provide reporting capabilities to groups. The charter was created as a result of functionality recently added to the application. It has never been run before, being a product of previous design and exploratory sessions. As such, the procedure has no fixed time limit, but 30 minutes have been set aside for this initial session. This is a paired test, with one tester running the application and the other observing, providing insights, and taking test notes as necessary. Note that the dialog between the testers has replaced the imaginary user dialog. Me: Are we set to run the procedure? Co-tester: Ok, let’s do this and then we can take lunch. Based on our surface analy- sis, we’ve identified some potential risk points in the login screen, but the permis- sion-based implementation they created should be straightforward. Me: I’m initially going to investigate the login accessibility risk. Hmmm … I can’t navigate out of the login area using the Tab key … make a note that the navigation requires the use of Ctrl+Tab. Co-tester: That company logo at the bot- tom looks kind of ugly. I’ll make a note to see if we can improve it. Me: Whoa! Things were going OK until I navigated back from the link. See the how the focus was moved to the top with no focus indication? Let switch back to mouse operation and try out the reporting assign- ment. They implemented a permission- based system based on user role. Co-tester: You know, there may be a lot of different users wanting this permission. This implementation may be too restrictive. I’ll make a note to investigate this further. Me: I’ve assigned the permission to the Do- main Admin. I’m logging in as that user Jeffrey H. Lucas is a Senior Software QA Analyst located in San Antonio, Texas, U.S.A. He has nine years of experience in software testing, and is the test lead on a small product development team. Prior to that, he has over ten years of test automation experience in other industries. His current interests are in the development of software test practices suitable for small development team environments. AUTHOR PROFILE - JEFFREY H. LUCAS and checking the permissions panel for the permission. Uh-oh, we have a problem. The indicator just uses a red-green color scheme with no alternative. That is defi- nitely an accessibility issue. Co-tester: Check your email. I configured the system to send you an email notifi- cation on change of permissions. Ouch! Do you see that link at the bottom? They should know better than to redirect to login screens through email – it can be exploited for spear phishing and whaling attacks. Me: I didn’t see that. I’m glad you were avail- able to assist with this. Software testing needs to incorporate intelligent review of oracles (such as stated user needs), risks, and user flows throughout the software development cycle. These two sessions illustrate some of the ways that the imagination and creativity of the tester can be leveraged: • Use charters to guide the test session, or specifically provide instruction that scripted procedures are guidelines only. If scripted procedures are necessary, then allow markup Continued on page 12 Figure 2: Permissions Dialog
12 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MindMap-TestingInProduction-http://buff.ly/PDCo23 Br ief His t orY OF Time a Fr enchEdition Continued from page 11 feedback from the test sessions that are reviewed regularly for incorporation. • Use paired testing to provide a “voice” to the thoughts of the testers. Additionally, this provides a second set of eyes that are not preoccupied with the mechanics of running the application. • Create the test sessions from previous session questions and ongoing risk analysis, instead of static procedures that are run blindly. • Use regular, short development cycles to analyze, validate, design, develop, and test the functions. The sessions should incorporate all players in constant communications instead of verifying requirements analyzed months before by people you have never met. In the case of the first session, the voices can be ignored for a while, but it comes to a point that they demand to be heard (either before or after delivery). Perhaps the better approach would be to get to know the end users of the product to champion their values and mindset by incorporating actual users (with real voices) in the process. (Chorus of voices) ... Amen to that... □ By Vipin Jain and Anubha Jain The buzzword is globalization. Shopping has taken the e-route with new ecommerce sites showing their presence each day on the World Wide Web. Credit card usage online has gone beyond unimaginable proportions. Each company has a web presence. Add multi-language support to this mix and a true globalized website is there. Wow! How easy is this or is there a catch? Web site globalization is as complex a process as it gets, and demands significant investment of time and resources. Oftentimes however the planning is poor and goals are not clearly defined; a wide range of technical hurdles and linguistic embarrassments easily stalls web site globalization. The solutions are non-trivial; however there are a number of “secrets” that may help you avoid some of the pitfalls of building and testing a multilingual site. Web globalization is still very much a practice one “learns by doing.” For many developers and businesses the thought of having multi language support within their portals might never have crossed their minds. They should however think of the benefits they will Continued on page 13 NEWSIN BRIEF Automation Samurai have just released a new QTP add-in that supports all versions of QTP. It utilises the latest Selenium technology (webdriver) to extend QTP features. With this Add-in QTP now can support the latest versions of Safari, Opera, HtmlUnit, Chrome, Firefox not only on Windows, but on all major platforms such as MAC and UNIX. http:/ /www.automationsamurai.com Testing Multilingual Websites Figure1.GoogleDriveMultilanguage www.testninjas.com
13 Follow us at www.twitter.com/testingclub CheckouttestingeventsfromMinistryofTesting-http://www.ministryoftesting.com/training-events/ Br ief His t orY OF Time a Fr enchEdition Continued from page 12 get from supporting multiple languages as this will likely lead to an increase in the rate of conversions, more user traffic and a wider global reach. When a multilingual site is planned and created, there is much to consider and it’s really all about localization and knowing the culture associated with the language. Although the practice of web production has matured rapidly over the years, the practice of web globalization is still very much in its infancy. In “Secrets of Web Site Globalization” John Yunker mentions that although nearly every US business now has a web presence, only 15% offer more than one language. With so few examples to build upon, the web manager planning a multilingual site is often left with little direction and support. There are many tips and things to take care of while developing a multilingual site; there are very few such tips for testing such a site. What should we as testers, do to make sure that the site displays the correct content for all languages?As an example, text in English has been embedded in the image shown here. Allwords,orkeys,arewritteninonecolumn.Thereareseparatecolumnsforeachlanguage.Each languagehasbeengivenacode,asspecifiedinISO639-2Code(http://www.loc.gov/standards/ iso639-2/php/code_list.php).WehaveusedGoogleTranslatefunctionwithintheGoogleDrive spreadsheet.ItssyntaxisGoogleTranslate(“text”,“sourcelanguage”,“targetlanguage”).Source languagehereisEnglish(en)andtargetlanguagesarementionedineachcolumn. FIGURE 1. NOTES Figure2.Google’sXMLversion Testenvironmentiskeyformultilingualtesting It is not enough to simply change the default browser language and perform tests in both the languages! Depending on how it is implemented, a web site may identify correct language for its interface from the browser language setting, the machines regional and language settings or a configuration setting in the web application. It is imperative that the web site be tested from various machines; each having different operating systems written in the particular language. The browsers and operating systems should be the ones that the stakeholders have agreed upon in initial product meetings. Translations should be correct Automated web translations always provide a good start and often translate correctly. However, nothing matches a native speaker of the language, belonging to the same region as the users. The translation she brings to the table will be the best when compared with other technological translations. It is a good idea to compare automated translations from multiple sources before using them in the test. Continued on page 14 VipinJainhasdedicatedlast10yearsofhisprofessionalcareertothesoftwarequalityand testing.CurrentlyworkingwithMetacubeSoftware,India,hedevelopedhiskeyskillsindevel- opingautomationframeworksandautomatingapplications.Withaprovenrecordofimple- mentingandrefiningtestprocessesforvariousclientsacrosstheglobe.heistheauthorof severalarticlesandsevenwellsoldbooksinIndia.Heisactivewithinthesoftwaretestingcom- munity,speakingatInternationalandnationalconferences,writingarticlesandcontributingto variousblogsandforums. AUTHOR PROFILE - VIPIN JAIN When we see the page containing this image in another language, the text on image still shows in English language. How to plan for such scenarios? After a few tips for testing multilingual sites, I will explain a simple but effective mechanism to plan for such tests. Know the cultural aspects A simple but challenging aspect of testing multi- lingual web sites is that each language might belong to a region that has a distinct culture. They may have a preference of colour (Spanish people prefer orange), text direction (people in middle east prefer right to left), format of salutations and addresses, measures, currency etc. are different in different cultures. Apart from language translation, other elements of the user interface should also be correct. Subscribe to Multilingual Compliance News
14 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MindMap-TestingandChecking-http://buff.ly/R127wk Br ief His t orY OF Time a Fr enchEdition Continued from page 13 Test Labels first Labels are the most static items on a web page. They are a great starting point for testing, as they tend to expand on translation. It is important to find any issues related to their truncation, overlay on/ under other controls, wrong word wrapping etc. Once this has been done, another area to focus on is error messages. The test should include generating all the error messages. If some text is not translated, three possibilities can exist. The text is missing, its English equivalent is present or junk characters show up in its place. A test model for multilingual sites Environment We have an application with support for 5 languages. The automation suite is written in Java. The test framework uses different property files for each language. These files are in XML format to be used by Java code. Challenge The challenge is to obtain the translation, maintain the translated text and generate correct property files in the form of XML. We are not going to purchase any expensive tools, but have used tools like Google spreadsheets and a couple of spreadsheet functions to achieve the goal. Just by the use of few simple spreadsheet tricks, we were able to generate properties files to be used with our automation suite. Manual translators are not available; hence we use Google translation service, which is the next best thing to manual translations. AnubhaJainisworkingasAssociateProfessor&Head,ITdepartment,InternationalCollege forGirls(ICG),locatedinJaipur,India.Anacademicianforlast11years,shehasbeeninvolved inteachingandmentoringseveralstudentsinthefieldofComputerScience.Knowingthat teachingisthebestformofgivingknowledgebacktosociety,sheworkedasalecturerin Subodhcollege,JaipurbeforesettlinginhercurrentroleatICG.Sheiscurrentlypursuingher PhD,infieldofInformationarchitecture.Anubhaistheauthorof9booksandaregularcontrib- utorinvariousforums. AUTHOR PROFILE - ANUBHA JAIN REFERENCES • http://www.loc.gov/standards/iso639-2/php/ code_list.php • http://support.google.com/docs/bin/answer. py?hl=en-GB&answer=1388877 Figure 1. shows what a Google Drive spreadsheet looked like. See figure 1. notes. Once this table is ready, our next step is to generate valid XML files, to be used by our automation code. (see figure 2.) This then needs to be copied for each language and once it is, all of the required XML will be ready. The automation script now needs to be tweaked. The code needs to identify the control named “Advanced Search” and click on it. Whatever language you are on, the control’s internal name shall remain as it is, as shown in the XML. Hence, our script shall run fine on all pages; irrespective of what language user has chosen. So with a couple of Google functions, and some basic coding skills, we can develop the data sets for each language and the framework developed to run our automation scripts is quick to recognize it. This will save all of us a lot of time in writing separate scripts for each language page. □ Using simple concatenation formula, we develop the XML within few seconds. =”<name=”&$B$4&”>”&C4&”</name>” Final XML looks like <?xml version=”1.0” encoding=”ISO-8859-1”?> <key-value> <name=Hello>¡Hola</name> <name=Advanced Search>Búsqueda Avan- zada</name> <name=ISBN List>ISBN Lista</name> <name=Author List>Author</name> <name=Title List>Lista de títulos</name> <name=New & Used Textbooks>Libros Nuevos y Usados</name> <name=Bestselling Textbooks>Libros más vendidos</name> <name=Bestselling Authors>Autores más vendidos</name> <name=Bestselling Titles>Los títulos más vendidos</name> <name=Used Books>Libros Usados</ name><name=ISBN Search>ISBN Búsque- da</name> </key-value> AUTOMATION CODE a community for software testers WWW.SOFTWARETESTINGCLUB.COM
15 Follow us at www.twitter.com/testingclub BugMagic-http://buff.ly/Q9FrPM Br ief His t orY OF Time a Fr enchEdition Figure1:JUnitOutput By Simon Knight It doesn’t seem like so long ago that I took to the Internet1 and vented about being frustrated with the limitations of the technologies we’d chosen for a big test project. Being a forward thinking kind of team, we’d decided to go ahead and utilise what was then the cutting edge behaviour driven design framework Cucumber, to help achieve our goal of rapidly identifying functional errors and regression issues using test automation, while building and deploying the software using the Hudson continuous integration environment, latterly Jenkins2 . Cucumber is a tool that can be used to support behaviour driven development using plain text feature files, which are then automated using Ruby. Features consist of scenarios, which in turn are made up of steps. Feature files and their corresponding scenarios are turned into automated tests by adding an additional layer of Ruby code, defining for each step what specific actions should be taken against the code under test – step definitions. Unfortunately it became quite evident from the early stages of the project that things weren’t really going to work out how any of us wanted. The original plan was for the intrepid tester (yours truly) to handle the test cases, or features in Cucumber speak, and then pass them along to the developers who would write the lower level implementations required to hook into the application code. This never really worked out how any of us had hoped or imagined it would though… The team of developers had their work more than adequately cut out for them in trying to deliver the necessary code to get the application up and running in the first instance. Implementing my test features was way down their list of to- dos, and with a full time exploratory testing gig, I simply didn’t have the time or knowledge of the underlying production code to be able to implement the features (lower level [Ruby] step implementations that hook into the production code) myself. Gradually the number of features left unimplemented grew, leaving the team feeling as though valuable time and effort had been wasted. Cucumber feature files without the corresponding step-implementations translated into considerable technical debt. Ian – technical architect for the project recalls – “There was a lot of frustration about tests SubSteps having been written but not implemented by the development team. It seemed like you in particular banged on about tests not being implemented for weeks on end - every single stand up meeting! Mixed feelings overall. The concept was good but the execution was very flawed.” We meet up in a pub down the road a little ways from our Sheffield office. Ultimately I wanted to talk to him about how we went from this place of supreme frustration for all concerned, to developing an alternative, ATDD (Acceptance Test Driven Design) style tool that met the needs of both testers and developers. BDD/ATDD Behaviour Driven Development (BDD) grew out of advances in Test Driven Development (TDD) and in particular the idea that unit level tests could be expressed in natural language rather than method declarations3 . Whereas BDD tests are likely to be owned (created & maintained) by the developer implementing the code under test, Acceptance Tests are more likely to be owned by the tester/subject matter expert. Jennitta Andrea defined ATDD as “the practice of expressing functional story requirements as concrete examples or expectations prior to story development. During story development… The story is “done”—deemed ready for exploratory and other testing—when these scope-defining automated checks pass.”4 Over mouthfuls of steak pie and real northern ale, he goes on to talk about the situation from his perspective: “The amount of time and energy required from the developers to code step implementations once tests had been scripted caused a lot of problems. It also seemed like there was a lot of repetition in the step implementation code - redundancy. I felt that it might be possible to expose the functionality in a more tester friendly fashion and move this work out of the developer domain thereby making it more manageable.” Did the other developers share your frustration? Ian - “To a degree. I’m not sure if I was being especially obstinate but I just wasn’t happy. I felt like it was slow and that it could be a lot of easier. “We were using Cuke For Duke originally which is a Ruby gem that bridges the gap between Java and Ruby, enabling you to write your step implementations in Java. The calls need to pass through the JRuby [version of Ruby that runs in the JVM] stack, into Ruby and then back out again ultimately into Java which is quite convoluted and a bit of a nightmare to debug - lots of information is lost. This may have changed now that Cucumber can be run as pure Java5 . “I was struggling to run Cucumber locally, in debug mode. I was using the original Ruby based implementation of Cucumber and to try and resolve the problems I was having I wrote a little Java application to run the steps for me. I realised at this point that there wasn’t actually a great deal of effort involved in converting the steps to Java instead of Ruby, and basically decided to keep going.” Did you experience any other problems? Ian - “My own coding experience was a bit of a limitation. If I’d studied computer science at university I might have had the opportunity Continued on page 18 SimonKnightisatesterbydayandEditorofTheTestingPlanetbynight.Organiserofocca- sionalBrummieTesterMeetups,Simonhasalsobeenknowntospeakandwriteonoccasion. Readhisblogherehttp://sjpknight.comorfollowhimonTwitter@sjpknight. AUTHOR PROFILE - SIMON KNIGHT
FOUNDATIONS OF CR
REATIVITY BY ANDY GLOVER
18 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MySurfaceChallenge-http://buff.ly/Q9FwTJ Br ief His t orY OF Time a Fr enchEdition Continued from page 15 to develop a compiler, which would have saved me some grief with the early SubSteps implementations. At some stage fairly early on I’d managed to code myself into a corner and had to rewrite the entire thing pretty much.“ Rewind back to the last few weeks of 2011. The project on which Cucumber had caused so much tension has been brought back from the dead and, like a kind of Frankenstein’s- monster-of-test; we decide to try to and splice both tools together. We end up with a set of legacy Cucumber features, and the beginnings of a new pack of tests – scripted, and implemented using the new tool and mostly without developer intervention. Both feature files and their step definitions can now: • Be specified and developed by the testers, • Be checked in and executed during code builds without consuming lots of additional development time. Leaving developers to focus on writing code, and the test team to focus on building and maintaining the automation library and providing additional feedback via exploratory tests. Paul - the Testing Manager - relays his excitement when he realised the power that had been bestowed upon him and his testers: “We actually started working on this a couple of years back while experiencing production issues with the Cucumber tests we were attempting to use on some of our products. Being able to script tests in the Gherkin Given/When/Then format was all well and good, but we still needed technical resources to implement the step definitions and turn them into executable tests. The development resources were never available and it didn’t make sense for our testers to become proficient in the underlying code as we use various languages depending on the project requirements. Gherkin is a Cucumber specific implementation of the ubiquitous language concept proposed by Dan North in his 2007 article, What’s in a Story?6 Features written in Gherkin steps normally follow the Given, When, Then format in order to define a set of acceptance criteria. “It took a little while and a couple of false starts for the project to come to fruition, but seeing my team start to use and experiment with our new tool has been an amazing experience.” So what kind of thing is it being used for? Paul - “One of the main benefits we’re seeing is vastly increased confidence in the builds that are being delivered to our testers. With 80%+ code [lines] coverage on one project and similar results on most others, regression errors are largely a thing of the past. At least in the QA builds they are anyway; with all of our tests running against the code every time a developer checks-in, problems are normally identified long before we get around to deploying a release.” Reclining into his 3rd floor meeting room chair, Paul is in the zone: “It’s a really efficient tool to use. Every member of my team can write automated tests that cover large chunks of complex functionality in just a few lines of code. Where it really shines though is in the maintainability of the test scripts. For example on one project we have a script that performs the following actions: Given I register a user And I login as an admin user And I activate the registered user And I logout as the admin user And I instead login as the registered user When I create a profile Then I can also create an organisation profile “An element was updated on one of the pages we hit during this test so we obviously needed to update the automation code to reflect the change. Because of the way in which our tests are written, the same action can be performed in many tests by utilising the same scenario step. This also means that for my change, I only actually needed to change 3 lines of code (the SubStep definition) in order to keep the test running. This change propagates out and is picked up by 12 additional tests that utilised the same scenario step (the call to the step definition). Total expenditure – around 15 mins to fix 13 tests; that’s not bad value for money in my book!” Since SubSteps’ inception, a decision has been made to open-source it so that it can be used and extended/improved upon by anyone that wants it. Although I’ve been using it for some time to build our project test capability, I’ve long since forgotten what was involved in getting it up and running, so I decide to give it another try. I navigate to GitHub and take a look at the SubSteps repository7 . Following the ReadMe file leads me to the project documentation and an example project. I download this onto my laptop (which already has Eclipse8 and Maven9 installed – you’ll need these if you try the same thing). Once I’ve unpacked and built the project using Maven I can import it into Eclipse and take a look. Right clicking on the FeatureFileRunner.java file and running it as a JUnit test opens up a Firefox window within which I can see the fairly extensive sequence of scenario steps being executed (these will also have been seen during the Maven project build). Within a matter of seconds, the example project selftest demo is complete. SubSteps is ready to roll. Let’s try scripting an actual test... I’ll need to create a new .feature file in the features folder – we’ll call it stc.feature and inside it I’ll create a test; a rudimentary check to see that when we login to softwaretestingclub. com, a welcome message is displayed. We’ll write the high-level outline in the standard ATDD/BDD Gherkin style. Tags: @all Feature: my first test Scenario Outline: A scenario to check I can login to the Software Testing Club website Given I am on the STC home page When I login with my email <email> and pass- word <password> Then I see a welcome message Examples: |email |password| |user@email.com|password| Done. But I also need to create a lower level .substeps implementation file. I’ll call it stc.substeps to keep things nice and simple. Inside it we need to define the steps that are actually taken to achieve Continued on page 19 Figure2:GreenTest
19 Follow us at www.twitter.com/testingclub Jobs!TestingJobs!Findthem.Postthem.http://jobs.softwaretestingclub.com Br ief His t orY OF Time a Fr enchEdition Continued from page 18 each line of the feature. As you might expect though; things don’t work out exactly perfect on the first go (see figure 1.) I run my test for the first time… A Firefox instance opens. Elements and pages are traversed onscreen. It closes. A fail. By the looks of the JUnit output, our first couple of steps passed, but the final step failed – the page assertion appears to be incorrect. However I can’t easily identify what happened because the browser window closed after the test and everything happened too fast to monitor. Let’s change that by opening the localhost. properties file and setting the visual.webdriver. close.on.fail=true flag to false. Now if we run it again, we can see that actually the test failed on the 2nd step - ClickById xg_tab_profile which doesn’t seem to get as far as opening up the next page because the click fails to register against the correct element. I pay a visit to the documentation via GitHub to try and resolve this. I can see that there are already quite a number of bespoke commands available for me to pick up and start using immediately. If there wasn’t of course, being an open source tool I could just write one myself. On this occasion I won’t need to though, since the FindByTagAndAttributes tag=”<tag>” attributes=[<attributeString>] command looks like it will do the job. Eventually I end up with the implementation below (also see figure 2.): Define: Given I am on the STC home page NavigateTo http://www.softwaretestingclub. com/ AssertPageTitle is “Software Testing Club - An Online Software Testing Community” Define: When I login with my email <email> and password <password> ClickLink “Sign In” FindById signin_email ClearAndSendKeys “<email>” FindById signin_password ClearAndSendKeys “<password>” FindByXpath //div[2]/div/div/div/div/div/ form/div/fieldset/dl[3]/dd/input Click Define: Then I see a welcome message AssertPageTitle is “Software Testing Club - An Online Software Testing Community” FindById welcome-text And… A green test! Clearly in a real-life scenario we’d want to expand upon this considerably and add a layer of manual exploratory testing, but it’s a great start. Over another drink, Ian shares big plans for his protégé: Where do you see the future of SubSteps? Ian - “We want to expand the number of features that SubSteps supports - enhancing the language, more selectors, a greater range of assertions. We’d like to be able to check and manipulate values in the database. We’d also like to be able to do more stuff with email. The [SubSteps] Eclipse plugin10 could do with some better editing functionality, the ability to run SubSteps tests more easily, better highlighting and navigation - stuff like that. It’d be good to integrate the tool with a reporting server so that we can get more information out and track things like what features are being run over time with pass and fail stats etc. “We’ve also considered the possibility of using SubSteps as a means for driving performance tests. We need to do some more research into this though as there is a feeling that there might be some limitations with the WebDriver API. Maybe we can swap in an alternative, more suitable API for achieving this at some point... We’ve looked at the possibility of using test-nodes that can run features with results being collated into a central repository - Selenium Grid style.” So what’s changed since my rant against Cucumber and BDD style testing last year? The main upside to using SubSteps for me is immediate feedback; my biggest gripe originallyi being the disconnect between delivering test cases and seeing them actually executed against the system under test. Over time, I’ve also experienced first-hand the confidence-gain that having a large and easily maintainable pack of rapidly executed BDD style tests can bring. Over the coming months we’ll be exploring ways in which the SubSteps platform can be leveraged to enhance our test automation stack still further. If you want to, you can get involved too by downloading, using and extending the code - here: http://technophobia.github.com/substeps-webdriver/ □ REFERENCES 1. http://sjpknight.com/birmingham-software- testing-club-meetup-15112011-simon-knight/ 2. See Jerry Schwartz’s excellent Jenkins article Continuous Integration for Testers last issue, TTP8 - July 2012. 3. http://dannorth.net/introducing-bdd/ 4. http://www.stickyminds.com/Media/eNews- Letters/Iterations 5. http://aslakhellesoy.com/post/20006051268/ cucumber-jvm-1-0-0 6. http://dannorth.net/whats-in-a-story/ 7. https://github.com/technophobia/substeps- webdriver 8. http://www.eclipse.org/ 9. http://maven.apache.org/ 10. Not yet publically available. www.testninjas.com
20 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Areyoupartofthefacelessmasses?http://buff.ly/Q9FDhU Br ief His t orY OF Time a Fr enchEdition Testing Smartphone Apps By Ulf Eriksson For many people, the use of apps has become an integral part of everyday life, but what is it that makes an app worth talking about? Why do some apps become world news, while others are never used? First of all, let’s decide what we’re talking about here. A mobile app is a piece of software that you download, install and run on your phone, much like software on your computer. A mobile web app, on the other hand, is an Internet-enabled app with specific functionality for mobile devices and which is accessed through the mobile device’s web browser, meaning that unlike mobile apps, they don’t need to be downloaded and installed on the device. Currently there are over a million apps1 and the number grows every day. There is therefore great demand for being able to test apps; but what should you think when you’re asked to provide quality related information on an app? Should you work in a similar fashion to testing any other software application, such as desktop software? Read to find out. Purpose is the key word When you are testing an app, it is most important to have an understanding of the purpose of the app. Perhaps it is even more important than when testing traditional programs and systems. You as a tester need to know what the app is supposed to be used for and what its target audience is. There are apps that are mainly functional; that is facilitating a function in everyday life. Let us take XE Currency as an example. It is an app that tells you the value of a chosen currency against the value of a number of others. Simple. The purpose of the app is to make life easier for travellers or indeed anyone who needs The target group also has a great importance for testing. If a bank is to develop an app, security will be a top priority because it relates to users’ money. An app that can find new music, however, needs to put more effort into appearance and user- friendly search functions, given that the risk of the user losing money through the app is very low. Therefore, it is important to prioritize requirements and tests based on the purpose of the app. Apps with clear requirements are the winners It’s not news that well-formed requirements are a prerequisite for successful testing. When it comes to apps, it is not uncommon for management to state that the app should have the same functions as the company’s website. In these situations, it is important that you dare to ask questions and state your requirements. It is crucial to get decision makers to understand that an app can not, and even should not, do everything that is done on the website. That’s what the web site is for. An app needs clear boundaries on what it is to do and what it is not to do. If it doesn’t have these boundaries well and clearly defined, it runs the risk of being too broad in scope and as a consequence, poor in functionality. An app should be limited to a few specialised functions and perform them well. As mentioned above, a well-defined target group is very important when testing apps. It is therefore very effective to use personas (descriptions of the proposed end-users) from the start of the project. This is so that clients, requirements managers, and testers have the same picture of the target groups the app is going to aim Continued on page 21 a quick and efficient look at the values of a bunch of currencies. To compare it with something completely different, let’s take Flick Kick Football: a game in which you flick across the screen to shoot and curve a football into the goal. The app is pure entertainment through a fun and different idea. The app does not need to fill any particular practical purpose. In this case, the look and feel plays a much greater role. In other words, the test strategy for these two apps differs sharply because of their purpose. In the case of XE it is important to verify the synchronization of information from XE’s currency exchange figures providers, while Flick Kick Football requires no connection to external systems, just seamless game play. AUTHORPROFILE-ULFERIKSSON UlfErikssonheadsReQtest;anonline bug-trackingplatformbasedinStockholm, SwedenandistheculminationofUlf’sde- cadesofworkindevelopmentandtesting. ReQtestisahandyandsimpletooltotrack bugs,listrequirementsand manageallcommunication byanyoneinvolvedinany project.Ulfrecentlyjotted histhoughtsontesting webappsusingmobile deviceshere-http:// www.reqtest.com/blog/ testing-a-web-app-us- ing-mobile-devices/ Figure1.XECurrency
21 Follow us at www.twitter.com/testingclub CarnivalforaTester-RememberingOla-http://buff.ly/RxfzIE Br ief His t orY OF Time a Fr enchEdition Continued from page 20 at. This makes it possible to adapt requirements and testing accordingly. One thing that is often forgotten when specifying requirements for apps is that underlying systems might also be affected by the app. A common example of this is that a popular app generates a lot of load and data traffic for existing systems. All of a sudden back-end systems might not be able to process the traffic, which in turn might negatively impact the business. Lesson - do not forget to specify requirements for the underlying systems. Think simplicity From a user’s point of view, an app has a lot to live up to. The app must be easy to use, since hard to use apps tend to be uninstalled immediately; after all a dull app is a dead app. As a tester, it is your duty to question design and functionality. The most popular apps usually have a very simple design and ensure that users immediately understand what to do to get started. Owing to this, usability testing is very important and has to be performed and prioritized when creating an app. Make it good The next important point is stability - an app that often hangs is not going to be accepted by users. Therefore it makes sense to carry out endurance tests and to verify that the app functions together with other programs. Furthermore, response time is very important. The app must not draw too much data traffic meaning that tests should be performed to ensure that the app only carries as much data that it needs, and only when it needs it. Performance testing at the server side should be carried out to verify that the performance is acceptable should the app rapidly become popular and garner many users. Test this in advance since popular apps often spread very, very quickly. We’ve all heard about apps which went from a handful of users to 100 000 in a couple of days2 . Last but not least, it should be easy to find the app. Because there is an incredible amount of apps you need good keywords. This is something that is easily forgotten, but that should be thought of when stating the requirements and then verified when testing the app, so that the app is found by the keywords specified in the requirements. The above might seem like a tall order, but the biggest challenge is to gain users’ trust and get good reviews in the app stores. Testers can assist in delivering an app with highest possible quality, but then it’s up to the user to decide if the particular app is useful to him or her. The mobile test environment A big difference in testing an app when compared to desktop applications is that the phone or tablet itself is your test environment. Normally there are emulators in the development environment that can verify much of the app, but just because it works in the emulator doesn’t mean it will work when the app is installed on a device. There is a flabbergasting variety of brands, models with different screen resolutions and versions of operating systems on phones and tablets. Once again, the requirements play a decisive role here. We need to know exactly which models, versions and operating system functions the app will support. You can also decide on a set of core functionality that must work on a huge variety of phones and then test on as many phones and devices as you can to ensure that your core functionality is seamless between devices. Incidentally, this is the way the BBC undertake their testing3 , and they call it their ‘core experience’4 . Furthermore, some apps interact with other features and applications in your phone, which will need to be verified. There may be features for Internet, GPS, messaging, call history and synchronisation functions. It is also likely there will be new versions of the app, so it is very important to perform upgrade tests. Ensure that saved settings and data do not disappear when a new version is installed. When the same functionality is to be tested many times on different units, it is natural to start thinking about test automation. To test apps automated via emulators is rarely a problem, but just as in manual testing, difficulties begin when you have to test against the real devices. The solution to be able to automate to the physical devices is to use some form of ‘device manager’ that the automation tool can communicate with. The device manager in turn communicates with an agent that is installed on the phone/tablet. That way, you can use the automation tools you are accustomed to, even when testing apps. Be aware of the risks Apps represent a relatively new technology that is constantly evolving and therefore there is very little documentation on how to test apps, which in turn may pose a risk. It is alarming how little we know about the apps we install on our phones or tablets. It may be clear when you install an app that it requires access to your messages, phone calls and personal information, but despite that, it is installed without hesitation. Safety testing has shown that 8% of the 10 000 most used Android apps “leak” information to unknown servers5 without your consent or knowledge and similar figures have been bandied about iOS. It is therefore very important to perform safety tests in order to minimize the risk that the user is exposed to attacks. Unfortunately, the majority of apps that are available are not well tested, which means that users will have to rely on what other people have said about the app. Summary We who work with quality assurance know just how important clear requirements are. Testing an app without a good specification is like building a house without a blueprint. If on top of that you don’t even know why or whom you’re building the house for, it will rarely end happily. Therefore, the following bullets are useful to accomplish successful testing: • The purpose, what should the app do and who is the target audience? • The requirements, they must be testable. • User-friendly, “less is more” when it comes to app design. • The test environment, do not forget to verify the app on a number of models and operating system versions. • Safety testing must be performed to protect the privacy of users. What makes an app worth talking about? I would say that an app that is stable and provides benefits in a simple, good or fun way is a given success. A proper mix of usability, design and functionality makes the users choose your app and spread the word. That in itself is the highest rating an app can get. □ REFERENCES 1. http://www.nytimes.com/2011/12/12/technol- ogy/one-million-apps-and-counting.html?_r=0 2. http://www.bbc.com/news/technolo- gy-17624172 3. http://mobiletestingfordummies.tumblr.com/ post/20056227958/testing 4. http://blog.responsivenews.co.uk/ post/18948466399/cutting-the-mustard 5. http://www.itworld.com/security/185485/study- shows-8-android-apps-leak-private-data- purpose Figure2.FlickKickFootball
22 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Atributetoagreatdearfriend-Ola-http://buff.ly/Q9FLhA Br ief His t orY OF Time a Fr enchEdition From Test to Driven to Design By Markus Gärtner The following could be a true story, yet it is completely invented. All characters and events in this article are entirely fictional. Phase 0 – QA does all the testing for us Tim Tester couldn’t await his new position at Big Data Corp. Inc. He had heard lots of rumours. Unfortunately his first day turned out to be a mess. He found several showstopper bugs; most of them could have been caught by lower level unit tests. When Tim walked over to Paul Programmer, he was told that the programmers at Big Data Corp. Inc. didn’t do any testing on their own. “It doesn’t say ‘unit testing’here.” Paul referred to his working contract. That’s why they have a separate QA department in the end. Tim knew that he had lots of work to do around here. When you are in the initial phase of the TDD adoption model, you write programs, and you don’t test them on their own. You rely on outside feedback about whether or not your program is working at all. This is often accompanied with hard to change code, suffering from accidentally introduced bugs, and being afraid to make any changes to the code. Often the code base is a big ball of mud, despite the best intentions to build a really good system. Testers are working overtime, project managers complain about the bad quality of the products, and during one-month customers report more bugs than the company can fix and deliver in the same time period. Eventually upper management figures out the problem, and fixes it, or maintains this vicious cycle by asking for more overtime or more staff. Phase 1 – Test automation Three months later Tim had convinced Paul to pair up with him. Tim explained to Paul how to write unit tests. “Every good unit test has three to four phases. Most often you will find some kind of Setup, then you execute some function on your tested object, and then you verify post- conditions on that object. When you involve database handling or other subsystems, you might also do a Teardown of your test. Since the former three phases are used in almost every unit test, people often refer to it as the Arrange-Act- Assert pattern.”1 Tim describes the basics of unit testing to Paul, including the different formats, assertions, how to write good assertions, and how mocking works. After Tim and Paul finished the work for the day, Paul felt confident enough to test his own code in future. Tim and Paul even found a multitude of bugs in code fragments that hadn’t been touched in a while. Up until today Paul had felt confident to use those code fragments and functions. Now he was more cautious. In the first phase towards test-driven development (TDD) programmers need to be able to write unit tests – good unit tests. Since any learning topic takes time to master, the initial unit tests will probably be bad, too descriptive, with obscure tests, test code duplication, and maybe they will even be fragile. That’s a good start, and you will probably learn something from it. At least you should strive to. Over time you will find out how to write good tests, and how to work around your code to be able to unit test it. But that’s not where you should stop, though. Phase 2 – Write the test first “Hi Paul, I just came back from a conference where folks talked a lot about writing the unit tests first, and then the code.” Tim was eager to share his newest insights with Paul. “Alright, so I write first a failing unit test, then I implement just enough to make that test pass, then do some refactoring, and start over? This sounds weird, let’s try it.” Paul and Tim worked all day on that new feature together, writing a failing unit test first, making the test pass by writing some tiny piece of production code, and removing all the duplications with the support of the passing unit tests in place. At the end of the day, Paul was unsure what to think about this new way of working. On the one hand, he saw benefits at places in the code where it was easy to drive the application. But what about GUI code? How would he write a failing test for the position of that button? And after all, didn’t this entire unit testing first thing lead to small but useless functions? Paul was unsure about the next steps, but his manager with the support from Tim kept on pushing. In the end, the quality of the product had improved greatly over the course of the past six months since Tim joined. When you have learned the basics of unit testing, you can start with writing a failing unit test before you actually write any code. At this point the move will likely feel uncomfortable, sometimes even painful. That’s ok. It’s just a sign that your learning journey has started. Keep on pushing forward. Over time you will gain enough expertise to eventually reach the next phase: Driving your development by writing a failing unit test. Phase 3 – Drive the Design Tim sneaked into Paul’s office. “What are you doing?” “I’m reading some of these online tutorials on TDD that you sent me. They explain that unit testing is not the point of TDD, but to drive the design of the application.” “What do you think about that?” “I think they have a point, but I don’t know how to do it.” “Let’s work on this together.” Paul and Tim worked on some more production code. Tim was able to show Paul more on design patterns, dependency injection and the SOLID Continued on page 23 NEWSIN BRIEF SeapineSoftwarehave launched QAWizard.com,a websitededicatedtoSeapine’s newQAWizardbusinessunit.QAWiz- ardoffers toolstohelpdevelopersand testersworkmoreefficientlyanddeliver workingsoftwaretotheir customers morefrequently. http://www.qawizard.com
23 Follow us at www.twitter.com/testingclub BecomeaTestNinja-www.testninjas.com Br ief His t orY OF Time a Fr enchEdition Continued from page 22 principles for good code. Paul did not understand everything at first, but at this point he had become used to learning new things whenever he worked together with Tim at the same screen. This time Paul learned how to drive his design with test-first development as well as how and when to separate his classes when they do too much work. Paul knew that it would take some more time to incorporate these lessons and be able to really work in a test-driven manner. After some time working with test-first development, you will realize there are good and bad ways to get started with TDD. You will also realize how to drive your code over several layers, and how to really use the TDD approach in order to help you develop your code in the long term. Dependency Injection2 , Coupling and Cohesion3 will become terms that you use on a daily basis. You will also know how to change the design of your application from one pattern4 to another5 , and how to use the support of your test suite naturally as an aid when making changes to the existing code base. Maybe a future generation of programmers will be able to work naturally from a failing unit test to the production code. Until then, we will have to remind ourselves to write a failing unit test for each piece of logic that we produce. But working test-driven is not the last step of the journey. There is room for still further improvement before we reach the goal. Phase 4 – Drive the Application outside-in “Say, Tim, I have a problem with that new functionality, and I don’t know how to fix it.” “Hey Paul – what’s the issue?” “You remember that specification workshop from two weeks ago? I am implementing that functionality for the table that we defined back then. It seems impossible. Where should I start?” “Ah, I see. Let’s start with the first example as failing acceptance test, and drive the application logic from that acceptance test.” “Can you show me?” Paul and Tim worked piece by piece from the acceptance test to the application layer. With each new step they reconsidered the growing structure for their support code, and tried to find out the code structure that was trying to escape the test code and become part of the production logic. Of course, each time they pulled some functionality out from the test code to the production logic, they made sure to either re-write the production code using TDD or to extract the existing structure, and retrofit enough unit tests to it. TDD is merely a stepping-stone towards driving your application from the outside in. All the other techniques that you learned in phases 1 through 3 are in place to support your critical thinking to drive the whole application from some high-level acceptance criteria. BDD enthusiasts might already find they apply phase 4; the rest of us need to go beyond what we have already learned, and learn how to extract application logic from test code, drive the application architecture and design from high-level examples, and keep them flexible to grow alongside with the application6 . Phase 5 – You don’t need code and tests anymore “Hi Tim, look, I found this article on model- based testing. They describe a framework that allows you to build the code and the tests from the same model. It’s open-source, and we won’t need to do that TDD stuff again with it.” “It’s open-source? Did you check whether they have unit tests in place? Oh, and did you find out how to test your model?” When you enter this phase, welcome to the future. You won’t need to write code and tests anymore, since there are tools for you to do that. But at this time your biggest problem probably is not writing code or tests, but the Y10k or the heavy flying car traffic around your space station. Of course this phase is sort of indistinguishable from phase 0, where you or your programmers didn’t write unit tests at all. After all, the least you should do, is to check whether the tools you are going to use match your expectations of 21st century coding practices – like including a set of automated unit tests. Of course, by looking at them, you could also spot whether these are phase 1 or phase 4 unit tests. Disclaimer The model that I went through in this article is by no means complete or intended to be a TDDMMi model, or something like that. It merely reflects my experiences with different workplaces, different programmers, and the whole idea for this article originated in a discussion back at ALE 2012 in Barcelona, Spain. You might have noticed that phase 5 is merely science fiction. After all, I would expect teams adopting TDD to take between one and three years to go through the first three phases. And these teams would produce leading edge code during that time. Please don’t rush through it, and like any model, don’t use it dogmatically. □ MarkusGärtnerworksasatestingprogrammer,trainer,coach,andconsultantwithit-agile GmbH,Hamburg,Germany.Markus,authorofATDDbyExample–APracticalGuidetoAc- ceptanceTest-DrivenDevelopment,astudentoftheworkofJerryWeinberg,foundedthe GermanAgileTestingandExploratoryworkshopin2011.Heisablack-beltinstructorinthe Miagi-DoschoolofSoftwareTestingandcontributestotheSoftwerkskammer,theGermany SoftwareCraftsmanshipmovement.MarkusregularlypresentsatAgileandtestingconfer- encesallovertheglobe,aswellasdedicatinghimselftowritingabouttesting,foremostinan Agilecontext.Hemaintainsapersonalblogathttp://www.shino.de/blog.HeteachesATDDand context-driventestingtocustomersintheAgileworld.HehastaughtATDDtotesterswitha non-technicalbackground,andhehastest-infectedprogrammersinseveraldomains. AUTHOR PROFILE - MARKUS GÄRTNER REFERENCES 1. TheArrange-Act-AssertPattern,http://c2.com/ cgi/wiki?ArrangeActAssert 2. MartinFowler,InversionofControlandtheDe- pendencyInjectionPattern,2004,http://martin- fowler.com/articles/injection.html 3. RobertC.Martin,AgileSoftwareDevelopment –Principles,Patterns,andPractices,Addison- WesleyLongman,2002 4. ErichGamma,RichardHelm,RalphJohnson, JohnVlissides,DesignPatterns–Elementsof ReusableObject-OrientedDesign,Addison- WesleyLongman,1994 5. JoshuaKerievsky,RefactoringtoPatterns, Addison-WesleyLongman,2004 6. MarkusGärtner,ATDDbyExample–APractical GuidetoAcceptanceTest-DrivenDevelopment, Addison-WesleyLongman,2012 BUGSIN THEWILD “Forthebenefitofpassengersusing AppleiOS6,localareamapsare availablefromthebookingoffice”  http://bit.ly/ios6tubemap -- Wrongturn:Apple’sbuggyiOS6maps leadtowidespreadcomplaints  http://bit.ly/anotherios6mapstory -- MistakewithStorify’sdatabase knocksdownstories  http://bit.ly/storifymistake
24 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag WhatisitliketobeaMozillian;Myfirsttwoyears.-http://buff.ly/RxfLYl The Evil Tester Question Time Provocative advice for testers who don’t know what to do! Listen to me at your peril IhavealwaystriedtodoExploratoryTest- ing,andfeltthatthatmademespecial. NowIhaveheardsomeonesaythatall testingisexploratory.AmIreallynotspe- cialatall,afterall? Q1. FROM Søren Harder Dear Søren, Ah, yes. It is hard when life slams our face into the pavement of reality. If you want to remain special, like me, then you have to stop listening to other people, like me. I like to use child logic on arguments that I want to treat as specious. Allow me to help you. I suspect that the definition of Exploratory Testing they use includes the null state. Then, even when the testing in question involves no exploration, it still involves exploration - it involves null exploration. You simply need to use a new, and more complicated scale, one designed to allow you to hang on to your delusion of specialness. I like to call this the ‘better’, or ‘best’, scale. Immediately discard the notion of null exploration as valid. You need to remove that if you want to be judgmental. I suggest initially setting your “fully exploratory” scale at 20%. Where 20% of testing is exploratory and 80% is not. Immediately you have become special again. And you have invoked the 80-20 rule, allowing you to make pretence of scientific categorisation. Then it becomes a simple matter of adding more grades in your scale to help you further label the testing of other people. There is one special step that I like to take. Enumerate everything that you do, and only you do, and then define ‘true’ Exploratory Testing as the specific combination of items that you enumerated. Then you not only cease ‘trying’ to do Exploratory Testing, you become the only person doing it. And that will make you really special. Helping peel faces off pavements since 2011, Auntie Evil Doyouhaveanytipsontakingthe<insert nominatedcertificationboardnamehere> certificationexam? Q2. FROM ANON Dear Anon, Why yes I do. And I will not say, “do not take it”. It is your money, consequently, your choice. I, for example, once bought an expensive jacket with sleeves that were too short, I still wear it, to remind myself that I should spend my money more wisely and buy jackets with longer sleeves. We all spend our money on daft things every so often, so don’t feel bad about that; or do, I don’t mind. Keep to the syllabus. These exams are not like school exams, they want you to pass. After all, you paid for training, if you fail, it makes the trainers look bad, and they don’t want that. Stick to the syllabus like a piece of discarded chewing gum. Parrot like an African Grey. Your job, when sitting the exam, is not to display learning, your job is to regurgitate whatever you were told or read in the syllabus. Regardless of what you think about what you were told, your job is to repeat it back. For multiple choice: don’t think “what is the right answer?” Instead think “what do they think is the right answer?” And watch out for double negatives in questions, these are a useful lazy way of writing questions that trip people up. For written questions you need to remember that the people marking use a set of guidelines. These guidelines tell the marker what type of phrases to expect to see and how many marks to give for each phrase. I harnessed this at university by running through the paper very quickly, regurgitating and parroting words and phrases from the syllabus in the form of a mind- map or outline for each question. If you don’t score it out, it becomes part of your answer. Then I would come back later and finish the answer with written English, knowing that I had probably already met the marking guidelines with the brain dump and that I could not run out of time. You paid. They want to pass you. Make it easy for them to pass you. Write down clearly and concisely whatever it was they told you. Engage your memory, not your brain. Yours Educashunaly, Professor Evil Br ief His t orY OF Time a Fr enchEdition Whatisthemostevilmetricthatisusedin softwaretesting,anddidyouinventit? Q3. FROM Steve Green Dear Steve, Any metric can be used to beat people over the head; as such much malfeasance can be done in their name. “Number of Test Cases” ranks high on the twisted scale because, even when not wielded, simply gazing upon it, casts a spell of malevolence. Consider. One has to posit the existence of a tangible physical entity called a ‘Test Case’ before one can even make sense of the words. This immediately puts the innocent reader in a state where such terrible things can become manifest in this world. Most testers have not read the “Experior Maleficarum”, they have not learned to recognise Continued on page 25
25 Follow us at www.twitter.com/testingclub RegularTesters,getintoSecurityTesting-http:/ /buff.ly/Q9G8Zl Br ief His t orY OF Time a Fr enchEdition Continued from page 24 ‘words of disreputable definition’, and even an Indagator trained in the “Rituale Exploro” can lapse when confronted with primitive heathen words of power such as this. As soon as one posits the existence of a test case it become natural to ask questions such as “how many do we need?” And worse, “how many have you done?” And then you start to think you haven’t done enough, given how many you need to do. And how you do you increase the “Number of Test Cases” done? By making it possible to ‘do’ them faster, by associating each counted ‘Test Case’ with a ‘Test Script’ - this allows you to add cheaper and more ‘Unskilled Testers’ to the project to convert ever more ‘Test Cases’ into the ‘done’ state. And as we all know, once a ‘Test Case’is done. It can never be undone. Otherwise your ‘test coverage’metric and ‘test progress against estimate’ metrics become invalid. Therefore we restrict the validity to the ‘phase’within which their state changes. And so we see that a simple 4 word Whencanapersonsaythattestingisnot requiredforaparticularproduct? Q4. FROM Yogesh Sharma Hi Yogesh, I like questions with flippant answers. So, of course the answer is “Whenever they want”. They can equally say “Bibble Bibble” whenever they want. But I suspect you want a less flippant and more scientific answer (Bwa’haha). My first Google search on “% of IT projects that fail” provided me with a scientific range of “62 - 68”%. I will make this complicated statistic easy for my readers by conclusively stating that 70% of IT projects fail. Therefore, I can conclude that 100% of people on 70% of IT projects can say “Testing is not required for this particular product” and they can hold their heads up high in the hope that the project was doomed anyway. Pretty good odds. Consequently we only have to concern ourselves with the 100% of people involved in 30% of IT projects. We all know that “you can’t test quality into a product” therefore I can use this to conclude that the product will either be quality or it won’t, so we can’t use ‘quality’ as a justification for testing. So basing my judgement on the preceding science; I can say: If a person has the power to cause the project to fail, then they can say “testing is not required”, at the point they make the decision to doom the project. Making rocket science look easy, Oncle E □ invocation leads to the primitive constructs which once were used in the savage times including “Test Phase”, “Test Scripts”, “Unskilled Tester” “Pass/Fail”. Brrr, I get chills just thinking about it. I did not invent this metric. I confess, I have used it, but I didn’t inhale. Frater Evil By Amy Phillips Much of a tester’s work involves telling people that they’ve done something wrong or missed something out. We’re often left chasing incomplete requirements or missing bug fixes. Our approach and outlook can be negative (well we do spend a lot of time expecting things to be broken). Often this is our strength but just sometimes this approach can go against you. I recently held a bug bash to quickly hammer down a load of bugs that weren’t getting prioritised, but that were causing a distraction. We’ve done this before to deal with a steady increase of bugs. This time it was to clear up exceptions in our server logs; mostly problems that didn’t affect the user hence our failure to prioritise fixing, but still a distraction from real problems. Developers love to write code but ask them to fix a bug and you’re on difficult ground. Yes they know it needs doing and yes they know it’s caused by their work but that still doesn’t make it fun. After all when was the last time you looked forward to doing the washing up? With a little bit of creativity you can break Unleash Your Creativity AUTHORPROFILE-AMYPHILLIPS AmyPhillipsisTestLeadatSongkick,a startupprovidingpersonalizedalertsabout livemusicevents.Shehas8yearstestingex- perienceatcompaniesincludingRoyalMail, TheGuardian,andYahoo!Amyispassionate aboutleanprinciplesandstrivestoenable developerstomoveasfastaspossiblewith- outcausingchaos. to focus on new approaches or ideas, but don’t allow them to get burdened down with a negative outlook. Leaving your tester mind-set at the door and turning something that would usually be seen as a chore into a fun, team-led activity can help you succeed. Give it a try next time you face a boring but important meeting, a problem or even just the daily grind. You may not always succeed but you’ll at least have some fun trying! □ the negative cycle and have a fun and productive bug bash (or anything else!). Here’s what we did: • Ask the developers to have some input into which bugs get fixed. Our bug-tracking tool allows people to vote for bugs but any simple voting system would work. Developers rarely get a chance to speak up about which bugs they think should be fixed, so this was a great opportunity for them to fix their pet peeves. • Make it visible. We took over a large whiteboard and generated lots of company-wide support. Use team meetings, newsletters and noticeboards as a way to publicise the event and generate interest. During and after the event put out updates and results. Above all stay focussed on the positives. • Make it fun – We decorated our whiteboard with balloons, this was a bug bash after all! Each bug had a sweet attached to it (to eat during or after depending on how much sugar that fix was going to require!) • Cake – never underestimate the power of cake! Problems and delays create the perfect opportunity NEWSIN BRIEF Sauce Labs is very happy to announce the general avail- ability of OS X, iOS, and Android in the Sauce cloud testing services. Mobile web and desktop web developers can now use our service to run their functional tests on iOS (iPhone, iPad), Android, Safari+OS X, Firefox+OS X, Google Chrome+OS X.  http://bit.ly/ttpapplesauce
26 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag SecondSignofMadness:LookingforTestTools?-http://buff.ly/Q9GdMF Br ief His t orY OF Time a Fr enchEdition NEWSIN BRIEF Perfecto Mobile, A Cloud- Based Mobile App Testing & Monitoring Platform, Raises $15 Mil- lion Series C http://bit.ly/perfectoc By Richard Edgren Good, effective software testing requires methodical hard work in combination with a creative effort. The creativity is needed to get many perspectives (outside explicit requirements), and also to find and judge effective methods for running important tests. There is much written about how to be creative, and a lot of general tips can be useful for testing as well (search for “creativity tricks”, or read any book by Edward de Bono). But even more important is the testers’ environment; creativity comes primarily where it is allowed, and fostered. Swedish philosopher Nils-Eric Sahlin has collected a basic set of environment characteristics1 that make a lot of sense when applied to the software testing profession: generosity, a sense of community, qualifications, cultural diversity, trust and tolerance, equality, curiosity, freedom of spirit, small scale. You are always part of creating this environment; let’s have a look at some of these from a testing perspective. Qualifications Knowledge is needed in order to take fruitful leaps into the unknown. Knowledge of test techniques and heuristics makes it a lot easier to apply effective testing. Knowledge of the technology makes it a lot easier to come up with test ideas. Knowledge of the domain the software will be used in will make it a lot easier to find important problems. Knowledge of the product will make testing faster, and will help you find interacting areas of interest. It is vital to have a generous environment where everybody learns and shares, preferably from different areas. Diversity Different points of view can givwwe radically new ideas when they blend. If all testers found the exact same things, there would be a lot of double work, and no synergy. By using group dynamics, we can take advantage of the strong and weak sides of each team member. For example it might be good to have a member of the test team that always only does what is stated in the online help. Or can a testing team be too diverse? Trust & Tolerance If the testers aren’t trusted, they will probably do just what they are told to. To be creative involves doing new things, which requires trust. Patience to wait for exciting things to happen is needed; trust that a lot of time also needs to be wasted. Creativity under pressure is difficult. Creativity in general is connected to allow ones’ self to make mistakes. This is especially easy in software testing, since mistakes can mimic end users behaviour, and thereby expose issues. “If you’re not prepared to be wrong, you will never come up with anything original.”2 In a test environment with detailed test scripts, it must be OK to side step the instructions, as well as follow them rigidly. Creative testing is about doing REFERENCES 1. http://www.fil.lu.se/sahlin/kreativitet/content. html 2. KenRobinson,http://www.ted.com/index.php/ talks/view/id/66 3. http://en.wikipedia.org/wiki/Flow_(psychology) An Environment for Testing Creativity RikardEdgren,humanistictestersince1998,specializedingeneralitiesliketestingeducationand exploratorytesting.Memberofthink-tankthetesteye.com.Co-authorofSoftwareQualityChar- acteristics,authorofTheLittleBlackBookonTestDesign.TestspecialistatQamcomResearch &Technology,Karlstad,Sweden. AUTHOR PROFILE - RICHARD EDGREN things differently; so different approaches should be embraced. Humour Humour is often about looking at things from a different perspective, to combine things that shouldn’t be. Laughter is fun and generates a good atmosphere where creativity can prosper. That is why you can laugh at developers’ mistakes, but also on missed points in a bug report, or the occasional duplicate bug that you already reported yourself. But most important: If it is fun to work, we do a better job. Discipline It must also be noted that the best ideas often come after a lot of hard work. It might be a bad example, but Edison tried 1000 times before he succeeded with the light bulb. So a creative environment can’t be a place where you just do what you feel like. It takes discipline to learn the product good enough, to learn everything that needs to be learned, and to execute also the boring tests that are needed. Diving into the details and the whole, not far from Csikszentmihalyi’s Flow3 , seems applicable for software testing at its best. Summary The ingredients in this creativity recipe seem simple (I added humour and discipline to Sahlin’s list), but still there aren’t many genuinely creative environments. Sahlin concludes that the unpleasant reason for this is that we, as humans, are driven by the seven deadly sins. I rather believe that many of us somewhere have lost the most important ingredient: Intrinsic motivation.An environment that encourages individuals, that’s where testing creativity can grow. □
27 Follow us at www.twitter.com/testingclub IsthereaRegression‘half-wayhouse’? -http://buff.ly/R0B7gC Br ief His t orY OF Time a Fr enchEdition HOT BLOGS Three Software Testing Books I’d like to read byEvilTester http://bit.ly/3testbooks -- Its official, I have the happiest job in america byMatthewSullivan http://bit.ly/happytesting -- Quality Insurance byMikeMeurs http://bit.ly/Qki0Qs -- The Future Of Mobile Testing bySimonStewart http://bit.ly/futuremobiletesting -- Why I Won’t Go Back byElisabethHendrickson http://bit.ly/iwontgoback -- The Great Agile Testing Quadrants byMaartenvandenEnde http://bit.ly/W5FwWc -- Testers and UX and That’s Not My Job byPeteWalen http://bit.ly/testersandux -- When in a rush, stop and look around byPekkaMarjamäki http://bit.ly/wheninarush -- Can You Really Have Too Many Tests? bySebRose http://bit.ly/toomanyteststtp -- Testing 2.0 byGoogleTestingBlog http://bit.ly/testing2point0 COMMENTCHAT DISCUSS By Karen Johnson Creativity. Mention the word and images of artists painting come to mind. The images conjured up around creativity seem a stark contrast from those of us more computer and business-minded, but the divide between business and creativity doesn’t have to be so wide. When we solve problems, whether those problems are resource constraints or we think of clever ways to test a product, I believe we are being creative: Create Mind Maps A mind map can be a fast and easy way to capture ideas. For me, mind maps seem to work best when my ideas are muddled together, meaning I haven’t quite figured out a structure or a relationship between my ideas. Mind maps seem to work incredibly well when I’m just starting to capture my ideas. As I build a mind map I’m better able to see my ideas take shape; I see subtopics form and cluster; I can see the relationships between ideas. And by identifying those relationships a mind map helps me to organize what I need to do.As an added bonus, I can colour-code ideas to see priority and to direct my focus. For mind maps that I build for my week’s tasks (yes I build mind maps for the week and often use them in client discussions), the color-coding can help me to see when I’m overcommitted. I frequently use color-coding to direct my focus for the day or for my one-on-one meetings. With a mind map, I can see at a glance what I want to accomplish this day or this week in relation to a larger goal. I can add a branch or a node and easily relocate or suppress and expose branches if I want to focus in a certain direction. My mind maps grow with me as my ideas mature, take form, and become clear. And eventually I see the whole presentation or a project or a week (or whatever I’m mind mapping) come together. There is something forgiving about drawing mind maps on my tablet that has helped me to use mind mapping much more than ever before. The ability to pinch and zoom, collapse and expose branches quickly and easily is helpful in a way I was not able to achieve on a desktop computer or with paper. Read One way to generate creative ideas is to read like a fiend and capture your thoughts as you read. Highlight sections in books. Take notes as you move throughout your day. Tag books, magazines and blogs for easy retrieval. Build your own sources of inspiration that you can turn to when you need a spark of inspiration. My office is packed with books. My iPad and Dropbox account have resources I turn to easily even when I’m on the road. If you read about creativity and most notably if you read books written by Alex Osborn (who coined the term brainstorming and authored the book “Applied Imagination”) or read Michael Michalko’s books (“Thinkertoys” and “Creative Thinkering”) or Edward de Bono’s works (author of “Six Thinking Hats”), you realize solving problems is a creative process. You don’t have to draw or paint to be creative. Use a Deck of Cards You never know how you can develop an idea from some seemingly unrelated source to your work. For example, in buying and reviewing two different decks of “creativity cards” one deck called the “Creative Whack Pack” by Roger von Oech and the “Thinkpak” deck by Michael Michalko, I was inspired to collect software testing heuristics (fallible guidelines or rules of thumb for ideas you can then apply to a problem1 ) into a format that can be printed on a deck of index cards2 . Continued on page 28 Re-Creativity:aTestingTipsSpecial Figure1.Beginningamindmap brainstorming mind mapping group mind mapping brainstorming variations 6-3-5 3-12-3 speed thinking pass it on reflective thinking alone time lateral thinking six thinking hats whack pack Phoenix Checklist R: Thinkertoys by Michael Michalko R:http://www.kaner.com/pdfs/ExploringExploratoryTesting.pdf emptying the mind influence alignment map note taking questioning methods & techniques
28 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag ApplyingTheLessonsofRapidTestingIntensive-http://buff.ly/RxgJUD Br ief His t orY OF Time a Fr enchEdition REFERENCES 1. TheEssenceofHeuristics,JamesBach-http:// www.satisfice.com/blog/archives/462 2. Anexampleofwhatsuchcardsmightlooklike canbefoundonAdamBrown’sbloghere:http:// testing.bananalogic.net/playing-cards/ 3. LynnMcKeealistingofheuristicsandmnemon- icshttp://www.qualityperspectives.ca/resourc- es_mnemonics.html 4. MichaelBolton,Article-TestingWithoutaMap -http://www.developsense.com/articles/2005- 01-TestingWithoutAMap.pdf 5. KarenN.Johnson,HeuristicCardDeck.Available asadownloadfrommybloghttp://karennicole- johnson.com/wp-content/uploads/2012/07/ Testing-Mnemonics-as-cards.doc 6. ElisabethHendrickson’scheatsheet-http:// testobsessed.com/2007/02/19/test-heuristics- cheat-sheet/ 7. ContextFreeQuestionsforTesting,Michael Bolton-http://www.developsense.com/ blog/2010/11/context-free-questions-for- testing/) 8. CemKanerandAndyTinkhamonthePhoenix Checklist-www.testingeducation.org/a/explore. pdf 9. ThePhoenixChecklist,RobLambert -http://blogs.imeta.co.uk/RLambert/ar- chive/2008/10/17/is-software-testing-like- being-in-the-cia.aspx Build Checklists Maybe a cheat sheet or a checklist doesn’t sound very creative but it could be. Consider a well-known checklist called The Phoenix Checklist, a list of questions developed by the CIA (Central Intelligence Agency). This is a list I became aware of by reading Michael Michalko’s book, “Thinkertoys.” Why not take a look at the list and see if it helps you. After all, there is value in standing back and seeing if your viewpoint changes, to gain a perspective you didn’t have when you were standing in the same place using the same point of view, trying to solve an issue. How do you use the Phoenix Checklist? Use the checklist like any checklist or heuristic, with judicial prudence. Look at the list. Find a question you can use and see how the questions help you consider a problem from a different viewpoint. I’m not the first tester to discover this checklist, take a look at Michael Bolton’s blog7 and how he used the checklist and made it his own by customizing it. Or how about the work of Cem Kaner and Andy Tinkham who found value in the checklist as well8 . Rob Lambert blogged about the Phoenix Checklist too9 . On the topic of checklists, a book called “The Checklist Manifesto: How to Get Things Right” by Atul Gawande; highlights the value of and use of several checklists. The book is a fast read and helped me to see that a checklist doesn’t have to be rigid, but can be a helpful reminder of items you might overlook when a situation becomes “almost too automatic.” Learn to Brainstorm The word brainstorming is used frequently and sometimes haphazardly. Sometimes it seems as though if two or more people are together and more than one idea surfaces, we say we are “brainstorming.” I’d recommend going back and reading about brainstorming to rediscover the original ideas first published by Alex F. Osborn. Osborn wrote several books and specifically I’ve been reading “Applied Imagination, Principles and Procedures of Creative Problem Solving” and “Unlocking Your Creative Power: How to Use Your Imagination to Brighten Life, to Get Ahead.” Osborn explains why judging an idea too early in the ideation process is a detriment. He illustrates why quantity over quality of ideas is important. Quantity in our critical society seems counterintuitive but Osborn believed that by furthering an idea or by expanding or combining ideas, we could arrive at great ideas. When we shoot down someone’s idea we stifle that person’s flow of ideas and the potential for what may come next. Osborn details how to host a brainstorming session and variations on brainstorming such as having a solo brainstorming session (something I do often as a frequent lone tester and independent consultant.) In delving deeper into heuristics and brainstorming, this path of research brought me back to a book (long referenced within the testing community) called “How to Solve It:ANewAspect of Mathematical Method” by George Polya. While the book on the outside may seem like a guide for math teachers, the list of ways to look at problems from analogies to puzzles has been shaping how I look at issues and most specifically testing challenges in a new light. Play a Game Breaking up problems into smaller problems or trying to bring a spirit of light humour to problems somehow brought me to read the book “Gamestorming: A Playbook for Innovators, Rulebreakers and Changemakers“ by Dave Gray, Sunni Brown and James Macanufo. This book offers a fantastic list of activities that can be used from ice breaking to brainstorming to bringing a group together. Some of the exercises do fine for one person and other activities a more suitable to a group. Summary Having small easy to use tools like card decks and checklists, and being able to use those tools helps me to think of testing ideas, approaches and solutions. With many creative ideas and tools at hand, you can inspire your testers or yourself, or be ready to ask good probing questions in a design session that may prevent a defect from ever being built. If you can do those things, then you’re innovating and you are creative. And you didn’t even have to draw or paint. □ KarenN.Johnsonisasoftwaretestconsultant.Sheisfrequentspeakeratconferences.Karenis acontributingauthortothebook,BeautifulTestingbyO’Reillypublishers.Shehaspublishednu- merousarticlesandblogsaboutherexperienceswithsoftwaretesting.Sheistheco-founderof theWRESTworkshop,moreinformationonWRESTcanbefoundathttp://www.wrestworkshop. com/Home.html.Visitherwebsiteat:http://www.karennjohnson.com AUTHOR PROFILE - KAREN n. johnson Continued from page 27 Using my heuristic card deck helps me. By picking up the deck and peeling through the cards, I always end up with multiple ideas and those ideas unlock my thinking. One idea leads to another, and the next thing I know I’m brainstorming a list of testing ideas. Not all heuristics work in all contexts but I believe most testing heuristics contain a word or a concept that just in mentioning or considering inspires a testing idea. If you read software-testing blogs, you will find many testers have written about heuristics and mnemonics3 4. A mnemonic is a device to help your memory; some testing heuristics have a mnemonic associated with the heuristic. These testers look for commonality or patterns in testing and find a premise they can use repeatedly and by doing so develop a heuristic. Why not tap into those ideas? Or look for patterns or situations in your own testing context and build a heuristic that makes sense to you. You could build your own card deck (mine5 is still a work in progress), or a reference sheet and in creating your own set of heuristics, you will have become more creative and furthered your own skill at solving problems. For example Elisabeth Hendrickson built what she refers to as a “two-page cheat sheet” of testing ideas6 . I find this list helpful. Her cheat sheet is packed with ideas. Figure2.ThinkpakbyMichaelMichalkoand CreativeWhackPackbyRogervonOech Figure3.HeuristicCardDeck
29 Follow us at www.twitter.com/testingclub FollowusonTwitter!@testingclub@testninjas@testingfeeds@stcjobs CREATIVE LEARNING MINDMAP Br ief His t orY OF Time a Fr enchEdition Creative Learning Traditional Learning Certification Degrees Training courses Through people Talk to people! Go to meetups Encourage internal activities Hack days Away days Paired activities / testing Online Communities Through resources Books, publications, blogs... Game Storming Software TestingClub Weekend Testing UTest Challenge Yourself Start a blog Make something Online Coursera Udemy Stanford Online Open University Read plenty Take notes Share what you learn Draw a picture Track Your Learning Journal / blog Write It Down MindMaps How Drawings Tools Calendars Blog Photos Writing Tools Evernote OneNote Set Goals Monthly Read something Research something Write something Article Blog post Book! Tools Approaches Ideas Yearly Attend, organise or do... Training Course Conferences 3-5 Years Vertical Progression Horizontal Chamge Start a business Go independent What's your career path? Meetups Khan Academy Videos Technical Managerial Consultant SME Coach / Trainer Freelance Contract Do something different Read Widely Read deeply Code Academy Podcasts Participate If you are loud, be quiet Take a different route home Talk to someone different Learn something new Write an article Create a video Focus or defocus Ask for help or help someone Go beyond your comfort zone Word Processor Pen and Paper ...many more! Don't rely on your memory alone
30 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag XSTUDIO XStudio is a free ALM/test management solution allowing to manage requirements/specifications, scrum projects, Automated/manual tests, campaigns and defects. An LGPL SDK is also included to interface with proprietary tests. www.xqual.com ----------------------------------------------- PARASOFT SOATEST Parasoft SOAtest automates web application testing, message/protocol testing, cloud testing and security testing. Parasoft SOAtest and Parasoft Load Test (packaged together) ensure secure, reliable, compliant business processes and seamlessly integrate with Parasoft language products (e.g., Parasoft Jtest) to help teams prevent and detect application-layer defects from the start of the SDLC. Moreover, Parasoft SOAtest integrates with Parasoft Virtualize to provide comprehensive access to traditionally difficult or expensive to access development and test environments. Parasoft SOAtest provides an integrated solution for: • End-to-end testing • Environment management • Quality governance • Process visibility and control. www.parasoft.com ----------------------------------------------- LOADSTORM LoadStorm – The lowest cost and easiest cloud load testing tool. Free account for 25 users. Test up to 100k vusers. Real-time graphs with key performance metrics. www.loadstorm.com ----------------------------------------------- BecomeaTestNinja-www.testninjas.com TEST TOOLS & SOFTWARE Br ief His t orY OF Time a Fr enchEdition THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory GEMINI Gemini brings versatile test management, bug and issue tracking to your team. Sign up to our cloud-based offering or install locally. Join the new generation in software project management with Gemini – no hidden extras or crazy pricing. 3 Users FREE – No Gimmicks – Full Edition. www.geminiplatform.com ----------------------------------------------- TESTRAIL TestRail – Test Case Management Software for QA and Development Teams. Comprehensive web-based test case management software to efficiently manage, track and organize your software testing efforts. http://www.gurock.com/testrail/ ----------------------------------------------- TESTLODGE TestLodge is an online test case management tool that allows you to manage your test plans, requirements, test cases and test runs with ease along with issue tracker integration. www.testlodge.com ----------------------------------------------- TESTOPTIMAL Model-based data-driven test design and test automation to improve test coverage, enable rapid response to changes and reduce test maintenance cost. www.testoptimal.com ----------------------------------------------- ENTERPRISE TESTER Enterprise Tester | Award-winning test management platform offering great features, support, and pricing. Enterprise Tester provides a flexible test management and execution feature set, full coverage from requirements to defects, dashboards and reporting, TQL, and a REST API. It integrates with JIRA, TFS, Enterprise Architect, Selenium and more. Plus, if you are a Not-for- Profit or running an Open Source project you can ask for a FREE community license. Get started for $10 or try Enterprise Tester FREE for 30-days: www.enterprisetester.com/try ----------------------------------------------- KALISTICK Kalistick gives testers a new solution to design efficient test strategies focusing on business risks. Our unique technology analyzes test cases footprints and functional changes to select the most relevant test cases. Discover how to move one step ahead in testing efficiency. www.kalistick.com ----------------------------------------------- TESTWAVE TestWaveisanextgenerationtestmanagementtool implementedasSoftwareasaService(SaaS).Itcan bedeployedinstantlyandyouonlypayforwhatyou use.TestWaveisdesignedforbothTestManagers andTesters,andprovidesrequirements,test planning,testexecutionanddefecttracking.Intuitive graphsreporttestingdatainrealtime.Reduceyour costsandunleashthepowerofSaaSwiththecloud’s firstfullyextensibletestmanagementtool. www.testwave.co.uk ----------------------------------------------- THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory
31 Follow us at www.twitter.com/testingclub CheckouttestingeventsfromMinistryofTesting-http://www.ministryoftesting.com/training-events/ Br ief His t orY OF Time a Fr enchEdition TEST HATS Test Hats are an independent software testing services provider, with offices in the UK and Spain. We provide a full range of testing services including System, Performance and Security testing along with specialised Consultancy and Training. For near-shore testing our Test Lab is fully equipped with a range of desktop and mobile platforms, testing software and tools, allowing us to provide a quality service at a competitive price. Visit our website to learn more about Test Hats and our services. Get in touch today to talk about how we can help test your projects. www.testhats.com ----------------------------------------------- THE TEST PEOPLE The Test People delivers the best, most innovative, highly technical and competitive performance engineering and test service available today. Based upon our extensive experience, TTP can deliver tailored services to address all aspects of the functional and non-functional test lifecycle, including highly specialised performance engineering and test automation services including automated build and continuous integration solutions. TTP are at the forefront of utilising the cloud for test and load environments, with significant experience in open source and the major commercial toolsets whilst also coming armed with our own performance and automation frameworks. www.thetestpeople.com ----------------------------------------------- REVOLUTION IT RevolutionITistheleadingQualityAssuranceand Testing,managementconsultingfirminAsiaPacific. WehelpourclientsdeliverITprojectsandhavecore offeringsacrossProjectManagement,Requirements ManagementandApplicationTesting.Wehaveover 250staffandofficesinMelbourne,Sydney,Brisbane, Canberra,AdelaideandSingapore.Ouroffering includesdeliveryconsulting,methodologies,tool solutionsandtraining.Wehavestrategicpartnerships withHPsoftware,IBMRational,Oracle,Agile AcademyandSAP.WithHPwehavebeentheleading HPSoftwarePlatinumPartnerfor4yearsrunningand theleadingreseller,1stlinetechnicalsupport,training andservicespartner. www.revolutionit.com.au ----------------------------------------------- EUROSTAR EuroSTAR is Europe’s premier software testing conference and has grown to become the largest and most prestigious event on the software testing calendar. Conference attendees can choose from numerous thought-provoking presentations, intensive tutorials, interactive sessions and inspirational keynotes. Plus, visit Europe’s largest software testing exhibition which showcases the leading companies in the industry. We hope you can join us for EuroSTAR 2012 in Amsterdam, The Netherlands from 5 - 8 November for the 20th anniversary of our testing conference! The EuroSTAR Software Testing Conference is where the testing community gathers for an intensive 3-4 days of learning, networking and discussion. www.eurostarconferences.com ----------------------------------------------- COMMUNITIES,CONFERENCES ANDNEWS www.testninjas.com WWW.MINISTRYOFTESTING.COM CO-CREATING SMARTER TESTERS EDUCATION • COLLABORATION EVENTS SOFTWARETESTINGSOLUTIONS THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory
OnthenightofOctober23,manyofuslostadearfriendandcolleaguewhenOlaHyltén,fatherof twodaughters,tragicallydiedinacaraccident.Throughouttheworld,Olawasknowntomanyin thetestingcommunityforhiskindness,witandhumour,aswellasforhissupportanddedication tothecontext-driventestingcommunity.Wewhohadtheopportunitytoworkwithhimwill alsorememberhimforhischaracter,strongsenseofethicsandunwaveringdesiretomakethe peoplearoundhimfeelgood.Olahadatruerock´nrollspirit.Heneverjustacceptedasituation becausesomeonesaiditwassoandhewasnotafraidtospeakupwhenhefeltheneededto. Ola’smottowhichhetrulylivedbywas“Ithinkformyself.”Manyofushavealsobeenpartnersin andbenefactorsofhisthinking. Olaworkedhardtoadvancetheunderstandingofexploratorytestingandrelatedpracticesathis workplaceinthelocaltestingcommunityandwastwiceaparticipantattheSwedishWorkshop onExploratoryTesting.Itwasduringoneofthoseworkshopsthattheideaoforganisinga Europeanconferenceoncontext-driventestingwasbornandlittlemorethanayearlater,the inauguralLet’sTestconferencewasheldinStockholm,Sweden,withOlaasconferencechair.As muchasOlalovedtestinghealsolovedmusicandespeciallyRock,sowhynotcombinethetwo ofthemhefigured.FewoftheparticipantsatthatconferenceislikelytoforgetOla’sopening remarkswhichstartedwithAC/DC:s“ForThoseAbouttoRockWeSaluteYou”asOlaentered thestage.Againatruerock´nrollspiritfromamanwithahugeheart.  JohanJonasson& HenrikAndersson Ola Hyltén 1966 - 2012 A Tribute To Ola

More Related Content

PDF
36532383 audiences-engagement-and-content-strategy-for-transmedia-storytellers
Robert Pratten
 
PDF
Participatory storytelling
Robert Pratten
 
PDF
Quest for Learning Engagement: Adventure Versions
Karl Kapp
 
PDF
Understanding Games and Gamification
Karl Kapp
 
PDF
Mission Possible: Creating Learner Engagement
Karl Kapp
 
PDF
TU204 - Beyond Gamification:Think Like a Game Designer to Create Engaging, Me...
Karl Kapp
 
PDF
PRINCE2_for_Fed_Gov_Projects
Richard Tucker, PMP, PRINCE2, CSM
 
DOCX
Script
lovelauren
 
36532383 audiences-engagement-and-content-strategy-for-transmedia-storytellers
Robert Pratten
 
Participatory storytelling
Robert Pratten
 
Quest for Learning Engagement: Adventure Versions
Karl Kapp
 
Understanding Games and Gamification
Karl Kapp
 
Mission Possible: Creating Learner Engagement
Karl Kapp
 
TU204 - Beyond Gamification:Think Like a Game Designer to Create Engaging, Me...
Karl Kapp
 
PRINCE2_for_Fed_Gov_Projects
Richard Tucker, PMP, PRINCE2, CSM
 
Script
lovelauren
 

Viewers also liked (18)

PPS
Cumple Romano
guestdf0761
 
PDF
Carta Recomendación Sofia Aldazabal Wood
Sof Aldazabal Wood
 
DOCX
Graficos informatica
Edgar Martinez Vazquez
 
DOCX
2 октября состоялся праздничный концерт ко дню учителя
serg32
 
PPTX
Efectos del ozono
LuzMoreto
 
PPTX
An science project
NancySpahr
 
DOC
Pp shooting schedule_template (1)
ShannonOrr
 
PPT
Desenvolvimento sustentável como instrumento para a paz
Fernando Alcoforado
 
PDF
A SIMPLE METHOD TO AMPLIFY MICRO DISPLACEMENT
ijics
 
PPTX
Vaibhava Kengeri
VBHC Value Homes Pvt. Ltd.
 
PDF
Love What You Do flyer
MCM Products, LLC
 
DOCX
Perfect attendance certificate
Thomas clarke
 
DOCX
Photo analysis partners
kirk128
 
PDF
Jugos que curan
Jorge Eduardo Galvis Gomez
 
PPT
Top 10 Property Management Pitfalls
guest7e5a12
 
PPT
H πιο κρύα περιοχή στον κόσμο
avramaki
 
PDF
IAM live 2016: Corporate Newsrooms // Case Study: AXA Winterthur
IAM Institute of Applied Media Studies, ZHAW Zurich University of Applied Sciences
 
PDF
Nuevo comercio internacional
Oxford Group
 
Cumple Romano
guestdf0761
 
Carta Recomendación Sofia Aldazabal Wood
Sof Aldazabal Wood
 
Graficos informatica
Edgar Martinez Vazquez
 
2 октября состоялся праздничный концерт ко дню учителя
serg32
 
Efectos del ozono
LuzMoreto
 
An science project
NancySpahr
 
Pp shooting schedule_template (1)
ShannonOrr
 
Desenvolvimento sustentável como instrumento para a paz
Fernando Alcoforado
 
A SIMPLE METHOD TO AMPLIFY MICRO DISPLACEMENT
ijics
 
Vaibhava Kengeri
VBHC Value Homes Pvt. Ltd.
 
Love What You Do flyer
MCM Products, LLC
 
Perfect attendance certificate
Thomas clarke
 
Photo analysis partners
kirk128
 
Jugos que curan
Jorge Eduardo Galvis Gomez
 
Top 10 Property Management Pitfalls
guest7e5a12
 
H πιο κρύα περιοχή στον κόσμο
avramaki
 
IAM live 2016: Corporate Newsrooms // Case Study: AXA Winterthur
IAM Institute of Applied Media Studies, ZHAW Zurich University of Applied Sciences
 
Nuevo comercio internacional
Oxford Group
 
Ad

Similar to The Testing Planet Issue 9 (20)

PPTX
Telling (User) Stories
Paul Goddard
 
PPTX
User Story Writing & Estimation For Testers By Mahesh Varadharajan
Agile Testing Alliance
 
PDF
The Mindset Change for the Agile Tester
TechWell
 
PDF
The Value of User Experience (from Web 2.0 Expo Berlin 2008)
Niko Nyman
 
DOCX
Do we really need game testers?
Johan Hoberg
 
PDF
Stc mag-feb2010
softwaretestingclub
 
PDF
Software Testing Club Magazine Feb 2010
Rosie Sherry
 
PDF
Using Storytelling to Improve Usability and Plain Language
Whitney Quesenbery
 
PPTX
A Story Rich World - UPA NYC - Sept 14
Whitney Quesenbery and Kevin Brooks
 
KEY
Power of Story
Whitney Quesenbery
 
PPT
Writing Effective User Stories
Janeve George
 
PDF
Playful Experiences
CHI Poland
 
PPTX
Experience design thinking (master 1)
Lisa Jukelevics
 
PDF
FROM CURIOUS TO CREATIVE
Brian Housand
 
PDF
The Agile Tester’s Mindset
TechWell
 
PPTX
Telling stories: Custom learning scenarios with immersive narratives and gami...
Brightwave Group
 
PDF
Digital storytelling part 1 writing
Pietro Polsinelli
 
PDF
We are THM
Three-Headed Monkeys
 
PPT
Growing software from examples
Seb Rose
 
PDF
Designing for Creativity and Kindness in Games
Mirjam Eladhari
 
Telling (User) Stories
Paul Goddard
 
User Story Writing & Estimation For Testers By Mahesh Varadharajan
Agile Testing Alliance
 
The Mindset Change for the Agile Tester
TechWell
 
The Value of User Experience (from Web 2.0 Expo Berlin 2008)
Niko Nyman
 
Do we really need game testers?
Johan Hoberg
 
Stc mag-feb2010
softwaretestingclub
 
Software Testing Club Magazine Feb 2010
Rosie Sherry
 
Using Storytelling to Improve Usability and Plain Language
Whitney Quesenbery
 
A Story Rich World - UPA NYC - Sept 14
Whitney Quesenbery and Kevin Brooks
 
Power of Story
Whitney Quesenbery
 
Writing Effective User Stories
Janeve George
 
Playful Experiences
CHI Poland
 
Experience design thinking (master 1)
Lisa Jukelevics
 
FROM CURIOUS TO CREATIVE
Brian Housand
 
The Agile Tester’s Mindset
TechWell
 
Telling stories: Custom learning scenarios with immersive narratives and gami...
Brightwave Group
 
Digital storytelling part 1 writing
Pietro Polsinelli
 
We are THM
Three-Headed Monkeys
 
Growing software from examples
Seb Rose
 
Designing for Creativity and Kindness in Games
Mirjam Eladhari
 
Ad

More from Rosie Sherry (9)

PDF
Why and How Testers Should Act Like Marketeers
Rosie Sherry
 
PDF
The Good and Words of Software Testing
Rosie Sherry
 
PDF
The Testing Planet Issue 7
Rosie Sherry
 
PDF
The Testing Planet Issue 10
Rosie Sherry
 
PDF
How Lean Is Your Software Testing?
Rosie Sherry
 
PDF
10 Reasons Why You Fix Bugs As Soon As You Find Them
Rosie Sherry
 
PDF
The Testing Planet Issue 4
Rosie Sherry
 
PDF
The Testing Planet Issue 2
Rosie Sherry
 
PDF
What made you a software testing leader?
Rosie Sherry
 
Why and How Testers Should Act Like Marketeers
Rosie Sherry
 
The Good and Words of Software Testing
Rosie Sherry
 
The Testing Planet Issue 7
Rosie Sherry
 
The Testing Planet Issue 10
Rosie Sherry
 
How Lean Is Your Software Testing?
Rosie Sherry
 
10 Reasons Why You Fix Bugs As Soon As You Find Them
Rosie Sherry
 
The Testing Planet Issue 4
Rosie Sherry
 
The Testing Planet Issue 2
Rosie Sherry
 
What made you a software testing leader?
Rosie Sherry
 

Recently uploaded (20)

PDF
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
PDF
Enhancing emotion recognition model for a student engagement use case through...
IAESIJAI
 
PPTX
Chapter 5: Probability Theory and Statistics
RandyFin
 
PDF
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
IAESIJAI
 
PDF
Getting Started with Data Integration: FME Form 101
Safe Software
 
PDF
Developing a website for English-speaking practice to English as a foreign la...
IAESIJAI
 
PDF
A review of recent deep learning applications in wood surface defect identifi...
IAESIJAI
 
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
PDF
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDCA Consulting
 
PDF
STKI Israel Market Study 2025 version august
Dr. Jimmy Schwarzkopf
 
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
PDF
Five Habits of High-Impact Board Members
OnBoard
 
PDF
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
PDF
Unlock new opportunities with location data.pdf
Precisely
 
PPTX
Modernising the Digital Integration Hub
Daniel Toomey
 
PDF
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
PDF
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Mindy Bohannon
 
PDF
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
Microsoft Dynamics
 
PDF
A novel scalable deep ensemble learning framework for big data classification...
IAESIJAI
 
PDF
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 
A comparative study of natural language inference in Swahili using monolingua...
IAESIJAI
 
Enhancing emotion recognition model for a student engagement use case through...
IAESIJAI
 
Chapter 5: Probability Theory and Statistics
RandyFin
 
Hybrid horned lizard optimization algorithm-aquila optimizer for DC motor
IAESIJAI
 
Getting Started with Data Integration: FME Form 101
Safe Software
 
Developing a website for English-speaking practice to English as a foreign la...
IAESIJAI
 
A review of recent deep learning applications in wood surface defect identifi...
IAESIJAI
 
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
Transform Your ITIL® 4 & ITSM Strategy with AI in 2025.pdf
PDCA Consulting
 
STKI Israel Market Study 2025 version august
Dr. Jimmy Schwarzkopf
 
Group 1 Presentation -Planning and Decision Making .pptx
MatthewLewis227954
 
Five Habits of High-Impact Board Members
OnBoard
 
Zenith AI: Advanced Artificial Intelligence
Biz Preneurs
 
Unlock new opportunities with location data.pdf
Precisely
 
Modernising the Digital Integration Hub
Daniel Toomey
 
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
A Late Bloomer's Guide to GenAI: Ethics, Bias, and Effective Prompting - Boha...
Mindy Bohannon
 
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
Microsoft Dynamics
 
A novel scalable deep ensemble learning framework for big data classification...
IAESIJAI
 
DP Operators-handbook-extract for the Mautical Institute
LeonelMejiaLarios
 

The Testing Planet Issue 9

  • 1. £5 By Lisa Crispin Humans have passed on his- tory and knowledge via story telling for millennia. Sharing our stories1 is a great way to share experiences that are likely to help others learn. Many of us have learned fam- ily lore or local legends sitting around a campfire or a dinner table. Those stories might have taught us something about how to be good members of society, or how to appreciate beauty. I myself learn best by example2 , so when I want to transfer ideas to others, I try to explain them via real exam- ples. Over the years that Janet Gregory3 and I have taught tutorials and classes, we real- ized that many people want to know: “Who else has had this problem that I’m having? How did they solve it? That solution might work for me.” When Janet and I started writing our Agile Testing book, we interviewed teams from around the globe, and asked them to tell us their stories. We found that many teams encountered similar obstacles, and were a bit sur- prised to see that many teams overcame these obstacles in similar ways. This felt like validation of our own experi- ence, and we included these stories as sidebars in our book. Judging from the feedback we’ve received from readers, many practitioners benefited Continued on page 5 By Martin Jansson and Greger Nolmark Testing is an activity that can be highly creative. In order to improve learning and testing we see improvisation as an important aspect. Improvisation can be done in many ways; the one we advocate is that of storytelling or role- playing. It is our belief that by taking aspects of storytelling into the world of collaborative testing we can increase our creativity. This is what we call an exploratory test adventure. Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing. We’re testing when we’re trying to find out about the extents and limitations of Continued on page 2 ALSO IN THE NEWS QA AND THE GREAT BOOKS To keep your quality as- surance chops fresh and to pick up new skills in... Continued on page 6 AN ENVIRONMENT FOR TESTING CREATIVITY Good, effective soft- ware testing requires methodical hard work... Continued on page 26 Telling OurStories FromTesttoDriventoDesign TESTING SMARTPHONE APPS For many people, the use of apps has become an integral part of ... Continued on page 20 Exploratory Test Adventure A Creative, Collaborative, Learning Experience November 2012 | www.thetestingplanet.com | No: 9 A fictional journey of a tester which might just sound very familiar - read all about it on page 22 Where will your testing lead you? THEEVILTESTER QUESTIONTIME More provocative advice for testers who don’t know what to do! Continued on page 24
  • 2. 2 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Aretesterscreative?Theactofcre- ationcouldbeseenasanactofgen- erosity;thetransformationofsome elementofyourpersonalityandintel- ligenceintoatangibleartifact.Creativity isvaluetosomebody.You,yourteam, yourproject,yourcommunity,theworld. The testing community is blessed with a great many creative personalities, writing books, running conferences and meetups, sharing their expertise by way of courses and seminars. Unleashing some intrinsic facet of their being into an unsus- pecting (but grateful) world. Some of their thoughts can be found in the fol- lowing pages, demonstrating that the creative spark can be ignited in many different ways and that the creative product can take many forms. Andlet’snotforgetthatThe TestingPlanetisthecreativeproductof themindsbelow.Wehopeyouenjoyit! Letter from the Editor Simon Knight K K Main story continued from page 1 the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before.” We begin with a definition of testing by Michael Bolton, quoted from the blog article Testing vs. Checking1 describing the extent of what testing could be. This definition highlights the aspects of the explorer or the adventurer, seeking new endeavours and finding new information about the world. We like the idea of being an adventurer who is out exploring searching for treasures, wearing the Indiana Jones kind of equipment, instead of the forgotten clerk behind the desk turning papers. One form of improvisation can be done through storytelling games. Here is one definition of story telling games: • A storytelling game is a game where two or more persons collaborate on telling a spontaneous story. Usually, each player takes care of one or more characters in the developing story. Some games in the tradition of role-playing games require one participant to take the roles of the various supporting characters, as well as introducing non-character forces (for example, a flood), but other systems dispense with this figure and distribute this function among all players. • Since this person usually sets the ground and setting for the story, he or she is often referred to as the “storyteller” (often contracted to “ST”) or “narrator”. Any number of other alternate forms may be used, many of which are variations on the term “gamemaster”; these variants are especially common in storytelling games derived from or similar to role-playing games. • In contrast to improv theatre, storytelling gamers describe the actions of their characters rather than acting them out, except during dialogue or, in some games, monologue. That said, live action versions exist, which are very much akin to theatre except in the crucial absence of a non-participating audience. This quote is taken from Wikipedia2 , which we see is fairly accurate in explaining what this is about. Notice the similarity with what we do when testing in a group, but where we as testers refer to it as telling a testing story. Something that is rarely seen in testing though is that the tester acts out in full the role in a specific situation. When testing like a specific user or trying to figure out what happens when encountering a specific event or obstacle, using the concept from storytelling games can help enhance the testing by letting it be more creative and if nothing else a lot of fun. When playing roleplaying or storytelling games you have a gamemaster or a storyteller. In testing we might instead have a coach, moderator, test lead or someone leading the exploratory test adventure. Just like storytelling, you rarely act on out in full when testing but instead simulate and act as if you were the user. What is an Exploratory Test Adventure? Participants As in traditional storytelling games we need to put together a number of participants with different roles. First we have the storyteller, in this case the test lead, coach or similar function. The storyteller is the one who leads the event and paints the main picture in which the participants should act in. Even though the participants should have some freedom to take the event to where they see fit the storyteller should always be there and guide them if they stray too far from the intended purpose. Secondly, we have the players, in this case the testers. As in collaborative testing the number of people participating can vary depending on situation. Thirdly we have other participants that for example might be observers and developers that can give and gain direct feedback and contribute with ideas on where to proceed in the adventure. Depending on what kind of team you have the participants and its observers will be differently composed. If you are in a cross-functional team, you should consider which roles the business analysts and the developers take on so that everyone step out of their comfort zone to increase the possibilities of the outcomes. Developers and business analysts could be observers of how the system or solution is actually used, how the participants play out a specific scenario. This could be an excellent way of getting early feedback on design and intended use. Scene This is basically the environment the participants are expected to act in. This could very well simply be the usual “test lab” but there might be some benefits to actually put the testers in a fictive situation. For example, the scene could be a pilot roll out with a certain department that is expected to use the application in a certain way. It is up to the Storyteller to create the scene and add background, actors and other details that are important to the participants. The more detailed the scene is, the better the creative experience will be for the participants. By bringing in events that might have happened in real life, the scene can become even more real. Replaying a scene that has happened to the storyteller can bring an interesting learning experience for all involved. Participants might do something completely different from what the storyteller did in the first place. Time In role playing and story telling games you usually know the time limit. It might be a short scene that you act in only meant to be played for a few hours. Then you might have what is called an adventure that could be played out in several occurrences. The last one is a campaign, which plays out during a longer time of many occurrences. We’ve only tried out the shorter versions when applying this to testing. It all depends on what you wish to learn and what you want to play out. The campaign timeline would fit when teaching students for a longer time, while the short hour version is applicable to events Continued on page 3 David Greenlees Thomas Harvey Simon Knight Rosie Sherry Michael Talks The team Hot Forum Post! Hiding bugs - http://bit.ly/hidingbugs Br ief His t orY OF Time a Fr enchEdition NEWSIN BRIEF Telerikrecentlyannounced the acquisitionof Fiddler, a WebDebugger,apopular toolcreatedtoinspectwebtrafficand ‘fiddle’withincomingoroutgoingdata.  http://bit.ly/telerikfiddler
  • 3. 3 Follow us at www.twitter.com/testingclub Continued from page 2 such as the test lab on conferences. Mission As with any game or adventure there usually is some goal that lures beyond the horizon. This gives a sense of motivation and pushes the players forward but it is quite common to not have a clear goal at the end of the road. In these cases it is the journey that is important. The end goal might very well change during the course of the adventure due to different events, for example the discovery of an area with unexpected critical issues. This will most likely shift the mind-set of the group. The ultimate goal might be obfuscated by lures that keep the participants astray or in confusion. Getting on the right path, as far as they know, can then be one of the sub-goals. There is an infinite number of ways this can be set up. Roles A storytelling game usually puts each player in a fictive character and this ties into the notion that testers could, or should, think in different ways. For testing we instead play a certain persona or role and act in their context. The available roles are closely connected to the scene created by the Storyteller. Something rarely seen in testing is the usage of character sheets when testing like a specific role or persona. A character sheet is a collection of background, skills, abilities and other useful information about the character. This could be made open to everyone or kept secret, these aspects and motifs of the personas will most likely lead to interesting discussions and reflections during debriefing. It can also contain information about how certain relations are between other roles or personas that might hinder or help the group. This might enable the tester to delve even deeper to understand the boundaries and limitations in which one character can act. Paths In an adventure the storyteller sometimes introduces an important intersection in the story. In some cases the paths look obvious, but it is up to the participants to try to choose their own paths. The storyteller should encourage creativity that may alter the path or break free in new directions that none had considered. In this way the whole group will be able to improvise in a situation that is unplanned and where it is uncertain where you end up. These unexpected twists and turns also gives the Storyteller a chance to hone their creative skills. Interrupts At any time during the exploratory test adventure the moderator may call for the attention of the participants, which pauses the scene. These interrupts may be pre-planned or invented along the way. One reason to do this could be to get the testers back on track if they deviate from set goals, another may be to just stop and do a quick reflection on progress so far. While yet another could be to zoom out and move an imaginary camera to another context to show something happening that will affect the participants, as a flare of bringing the story onward or letting them know something wicked is coming their way. Reporting In some role playing games the participants keep a journal on what happens, who said what and new plots that appear. Over time there is a need to keep track on the many events. This is quite similar to how sessions or threads are documented in SBTM or TBTM. The most interesting result is when many parties tell their story in these journals, making it obvious that everyone experience different things. Events The moderator may have certain events pre-planned that are designed to trigger new ways of thinking within the group. An example of this could be that the moderator describes what is going on during a meeting elsewhere between Project Leader and Manager that most likely will influence how the group acts. Another example is to have an invited guest come along with a different agenda to disturb the groups testing session. There can always be ulterior plots and events that are happening that might enhance the adventure or set obstacles in the way of the participants. Examples of Exploratory Test Adventures Let’s Test Lab 2012 In the Test lab there were three tables that focused on one system each. Each table also was a team led by one team lead. The first objective given out by Martin and James was to do collaborative test planning in the teams finishing with a debrief on what they had come up with. At this stage there were no interruptions. The next part was actually Follow us on Twitter! @testingclub @testninjas @testingfeeds @stcjobs Br ief His t orY OF Time a Fr enchEdition Martin Jansson, owner and consultant at Testverkstaden Sverige AB, started his career as tester 1996. He has tried many professions in product development, but his heart and soul belongs in testing. Mar- tin is one of the founders of www.thetesteye.com which has grown into one of the greatest Swedish blogs on software testing. Martin is a frequent runner of the testlabs at various test conferences. The last years Martin has assisted clients in evolving their organizations from a traditional one into an agile, where he focuses on what aspects of testing that works in an agile environment. You can reach him on Twit- ter @martin_jansson or on martin.jansson@testverkstaden.se AUTHOR PROFILE - Martin Jansson investigating what they had planned. During this session Martin interrupted and introduced a scene in which the test lead appeared on his way to a release meeting. He had just a few minutes before he needed to attend, so each team needed to quickly create a report on the most important issues and present them to Martin. The diversity in how the teams presented was very interesting. Some of the team members and some of the listeners thought the learning experience was great. The Test Lab was filling with over 50 people. The original plan was in the trunk. Each table consisted of at least 5 testers, and then there were many who stayed in the background listening and discussing what was going on. The scene was set by stating that they were testing teams, Martin and James took on the roles of project manager, line manager, test leads and CEO. Martin and James had from the beginning stated that they would go around the tables and say different things, thus contradicting each other. Present were also Ilari Henrik Aegerter and Anne-Marie Charrett who did coaching. There was great confusion in the teams since they were split between the objectives given out by Martin and James, while they at the same time wanted to work on their own agendas and while Ilari and Anne-Marie were trying to get their attention in coaching. All of us were improvising; the scene (what we thought it was) was changing and evolving all the time. During the whole session there were a few interruptions that presented the scene and its actors with new information, on which they could act on. During one of these interrupts Martin, as a project manager, asked why the teams were using the coaches, they had so much testing to do and so little time! This changed the team’s focus and removed some of the confusion. The experience was very different for the participants, but it seemed many enjoyed it and some learned more than others. (See3 for further details) Continued on page 4 Character Sheet: Local IT Support Background: Working at Customer IT Dept. Has been assigned to Acceptance Test Team due to high experience with previous releases of similar products. Focus/Interest: Tech - Install, Database, Performance Secret Plan: This release will most likely threaten the need for Local IT support. This due to plans to outsource support in connection with this launch. You are looking to discredit the product in any way possible.
  • 4. 4 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Hot Forum Post! Exploratory Regression Setup? http://bit.ly/exploratoryregression Continued from page 3 Other disciplines that could inspire us If we look into the world of organisation theory, there are movements that also investigate ways to become more creative. Here are a few quotes from the article Improving Case Discussion With an Improv Mind-Set by Andy Aylesworth4 that align well with our thinking of improvisation in testing. “To understand better and explain the phenomenon of organizational improvisation, scholars have turned to improvisational arts, specifically theatre and jazz, where improvisation is the norm, rather than the exception.” “Improvisational theatre has been invoked to better understand business innovation, management, collaborative technology, and team performance.” “... Employees to attend improvisational theatre classes, believing that such classes will lead to better, more productive, and creative employees.” By improvising they can explore a concept and try new ideas to better understand and perhaps learn something new. The concept comes from that of a Case study, which is very like that of running a scenario or setting up a scene to act in. The author thinks the use of improvisation inspired from the theatre would increase creativity when using the Case study technique. He also believes that participants, who usually are either rowdy or shy, might step out of their usual role and act in a different way. Participants in an exploratory test adventure would probably have a similar experience. When you take on a role that you might not otherwise have experience from, you need to consider new things and explore the boundaries in which you can act. In this situation your creativeness is key. The book Body and Language: Intercultural Learning Through Drama edited by Gerd Bäuer5 highlights several aspects regarding improvisation that aligns well with our ideas on exploratory testing adventures. Here is a section that is especially interesting from that book: “At least until computers can recognize Br ief His t orY OF Time a Fr enchEdition Greger Nolmark, Senior Test Consultant at Adecco IT-Konsult, started working with test in 1999. During the years he has had several different positions in the areas of test and support. His primary focus has been on testing always keeping an eye on the end user perception. Dur- ing recent years he has nurtured an interest in how new and different test techniques can help teams and testers to test better and to show there are more to testing than being certified. You can reach him on greger.nolmark@adecco.se or Twitter @GregerNolmark. AUTHOR PROFILE - Greger Nolmark to the exploratory test adventures is that of Experiential Learning. Gerald M. Weinberg has recently written a book about this, but you can also find lots of interesting blogs and such on the topic as well. David A. Kolb has researched a lot on this topic. “David A. Kolb (with Roger Fry) created his famous model out of four elements: concrete experience, observation and reflection, the formation of abstract concepts and testing in new situations. He represented these in the famous experiential learning circle that involves (1) concrete experience followed by (2) observation and experience followed by (3) forming abstract concepts followed by (4) testing in new situations (after Kurt Lewin). It is a model that appears time and again.” A quote from an article by Smith, M. K. (2001) on ‘David A. Kolb on experiential learning’6 . Experiential learning goes hand in hand with that of an exploratory test adventure. By setting up a scene in which we act, with events and encounters that simulate our everyday experiences we will learn new things and enable us to be creative in its context. Conclusion To use improvisation to be more creative and to enhance learning is not a new concept, still it is not often seen as part of people’s repertoire. Part of the reason is probably that many testers for different reasons work alone, instead of collaborating. Exploratory test adventures prepare the testers for the unknown by forcing them to go together through various simulated experiences where they need to improvise, which in turn nurtures creativity. We can even say that the foundation of a storytelling game is improvisation and creativity. This is why we promote that you, as a test lead, team lead or team member, step up and take the role of the storyteller. Play out a scene in which your team members can play an exploratory test adventure! □ REFERENCES 1. Bolton, M. (2009). ‘Testing vs. Checking’. Retrieved 2012-09-14 from http://www. developsense.com/blog/2009/08/testing-vs- checking/ 2. Wikipedia. ‘Storytelling games’. Retrieved 2012-09-14 from http://en.wikipedia.org/ wiki/Storytelling_game 3. Jansson, M. (2012). ‘A Let’s Test Lab Story’. Retrieved 2012-09-14 from http://thetest- eye.com/blog/2012/05/a-lets-testlab-story/ 4. Aylesworth, A. (2008). ‘Improving Case Dis- cussion With an Improv Mind-Set’. Retrieved 2012-09-14 from http://jmd.sagepub.com/ content/30/2/106.full.pdf 5. Bräuer, G. (2002). ‘Body and Language: Inter- cultural Learning Through Drama’. Retrieved 2012-09-14 from http://www.google.se/book s?id=xa2xD9pHonIC&printsec=frontcover&hl =sv#v=onepage&q&f=false 6. Smith, M. K. (2001). ‘David A. Kolb on experi- ential learning’, the encyclopaedia of informal education. Retrieved 2012-09-14 from http:// www.infed.org/b-explrn.htm and represent aural human speech a lot better than they can now and can be programmed to respond spontaneously to speech (which I, for one, don’t believe will ever happen), one cannot learn to creatively engage in a conversation in a language unless one has real human beings to interact with. Audiotapes and computer language programs can help one learn certain common exchanges or routine phrases, but to learn how to improvise new utterances one has not yet heard, at least one other speaker of the target language is needed. This is why informal improvisation drama activities are so powerful in the foreign-language classroom. To participate in an improvisation, one needs to use the body not only to produce appropriate language but also to express emotion and ideas through gesture, posture, and facial expression. Because the scene in a drama is an imaginary one, the participant is free to exaggerate or assume a persona that frees him or her to experiment with a wider range of language that ordinary exchanges might evoke. Improvisational drama is effective because of the repeated pressure it puts on participants to respond. It is not enough for students to hear the target language spoken; they need to talk themselves.” Notice the implication that by stepping out of your everyday situation into a context where you are not familiar and where your usual boundaries lie, you instead visit a world that can be explored in many different ways. All participants are part of exploring this new world and act in it according to their own set of beliefs and objectives. We see that the complications in learning new languages and the experimentation using improvisation align well with the complications of creating an environment that enables creative testing. Cem Kaner has expressed the complexity of teaching testing and many others have argued about the lack of actual knowledge gain by merely studying a curriculum such as that from the ISTQB. We see that there are similarities in teaching to express yourself in a new language with that of performing creative, context-driven testing. As you might figure out, improvisation and following a script does not go hand-in-hand. Another concept that is very applicable Character Sheet: HR Department Background: Working at HR department with recruitment. Has been assigned to the Ac- ceptance Test Team due to experience with similar systems. Focus/Interest: UI – Layout, Spelling, Error handling Secret Plan: Quick and dirty testing today. Real vs Barcelona on the telly tonight.
  • 5. 5 Follow us at www.twitter.com/testingclub REFERENCES 1. http://storycorps.org 2. http://exampler.com 3. http://janetgregory.ca Second story continued from page 1 from these experiences of other teams. Testers want to hear the real-life experiences of other testers. It’s simply another form of passing on wisdom via story telling. When I meet a new teammate or colleague, I want to hear their story. I might ask, “How did you come to be here?” Always, we discover we have several shared experiences. If we’re working together to build a high-quality software project, this helps us build a foundation to collaborate effectively. I like to hear stories from people with dif- ferent points of view. Each person who participated in an event tells a different story about it, depend- ing on their context, their perspective, and their experiences. We need all these different viewpoints in software projects, too. Each viewpoint has equal value. The diversity of viewpoints fleshes out the “big picture”. When your team must deliver a new fea- ture, ask a customer to tell her story with respect to that feature. Ask for examples of desired and unde- sired behaviour. Don’t constrain them, let them cre- ate! In my experience, many of us learn well from concise examples. We can use the same technique to specify how a software feature should behave. We can use all the different perspectives to help us think of more ways to explore and test the feature. Storytelling is a creative process. Software development (including testing) is also a creative process. Combine the two, and you have powerful ways to understand what your customers desire, and to make sure those desires are fulfilled. □ AUTHORPROFILE-LISACRISPIN LisaCrispinistheco-author,withJanet Gregory,ofAgileTesting:APracticalGuide forTestersandAgileTeams(Addison- Wesley,2009),co-authorwithTipHouseof ExtremeTesting(Addison-Wesley,2002), andacontributortoExperiencesofTestAu- tomationbyDorothyGrahamandMarkFew- ster(Addison-Wesley,2011),DevOpsfor DevelopersbyMichaelHuetterman(Apress 2012),andBeautifulTesting(O’Reilly,2009). Sheenjoyssharingherexperiencesviawrit- ing,presenting,teachingandparticipatingin agiletestingcommunitiesaroundtheworld. Shealsoenjoyslearningbetterwaystodeliv- erqualitysoftwareworkingasatesteronthe PivotalTrackerteam.FormoreaboutLisa’s work,visitwww.lisacrispin.com.@lisacrispin onTwitter,entaggle.com/lisacrispin Stories The power of story-telling Tell me your story How did you come here? Stories are examples We learn from examples People value "real-life" experiences Ask a customer to tell his/her story Helps understanding Can be used as example & tests Desired behaviour Undesired behaviour Different participants in an event tell a different story Each viewpoint is valid Diverse viewpoints "flesh out" the "big picture" NEW BOOKS Explore It!: Reduce Risk and Increase Confidence with Exploratory Testing By Elisabeth Hendrickson http://bit.ly/ehexploreit  Tap Into Mobile Application Testing By Jonathan Kohl https://leanpub.com/ testmobileapps Cucumber & Cheese - A Testers Workshop By Jeff Morgan https://leanpub.com/cu- cumber_and_cheese ATDD by Example: A Practical Guide to Acceptance Test- driven Development By Markus Gärtner http://amzn.to/WD5IWI Br ief His t orY OF Time a Fr enchEdition Figure1.StoriesMindmap Training Course: Coaching Testers - http://bit.ly/coachingtestersmot Br ief His t orY OF Time a Fr enchEdition
  • 6. 6 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Training Course: Techniques for Agile Manual Testers - http://bit.ly/agilemanualtesters QA and the Great Books Br ief His t orY OF Time a Fr enchEdition Brian J. Noggle has worked in quality assurance and technical writing for fourteen years in a variety of industries and situations. He currently works as a freelance software-testing con- sultant through his own company, Jeracor Group LLC, and specializes in short-term projects with limited budget. Noggle’s novel, John Donnelly’s Gold, is a comic romp in the world of IT. Although it be classical literature yet, he thinks you should read it. AUTHOR PROFILE - BRIAN J. NOGGLE By Brian J. Noggle To keep your quality assurance chops fresh and to pick up new skills in the technology field, almost every QA professional reads in the field. This includes blogs, Web sites, magazines, and white papers or longer by luminaries in the field such as James Whittaker, James Bach, or Cem Kaner. These sources can keep you up to date on the latest trends and thoughts in established quality practices and methodologies, but ultimately will only lead you to thoughts other people have thunk first. While you can successfully apply these quality precepts to your situation and can combine them with your experience, if you really want to become a thought leader and to make greater cognitive leaps, you’ll need to provide material for that great creative synthesizer between your ears. When you read books from outside the field, you imbibe ideas that you can integrate with your industry knowledge to invent new techniques and new ideas. While the impact might not be immediate, all information you take in becomes fuel for the creative fusion process. Years after you’ve read something, its lesson can come unbidden when you’re faced with a new problem or the same old problem in new circumstances. Or maybe the colour of the sky will remind you of the scene from a novel that will lead you to apply a situation from that book to your current project with significant results. If you’re game and want to challenge yourself (only a little bit, since most classical literature is actually pretty approachable if it does not intimidate you), I have some suggestions for a reading list. Books To Motivate In the quality assurance world, the whole de facto development process seems aligned against us. Every deadline seems achievable to management as long as the team can take those extra days out of the test cycle. The defects you’ve found, analysed, and advocated (the correction of, not the defects themselves) get ignored, put off into the never-arriving future, or otherwise removed without remediation. You return to your cubicle after the meeting and ask yourself, “Why am I doing this?” Sometimes you ask that when you get to your desk in the morning. Classical literature is rife with heroes who encounter difficulties in their day-to-day and carry on. The Bhagavad-Gita tells of Prince Arjuna, a member of a royal house who is off to war with some other royal houses, some of which are family. Arjuna doesn’t want to go to battle. His attendant Krishna explains to him how the world works and explains to Arjuna that it’s his duty and role in life, so Arjuna eventually does. If you’re looking for a little more Western perspective, pick up a piece of classic noir fiction by one of the masters of the 1930s and 1940s, whether it’s Raymond Chandler or Dashiell Hammett. Throughout their books, world-weary private investigators try to bring a little justice to the world by solving their small cases. Frankly, these books are closer to software quality assurance than they should be, with our need to make the world a better place through isolated victories in education or process improvements. Sometimes, though, a sap to the back of the head doesn’t sound too bad compared to another lessons learned meeting. Books About Time Management and Self-Improvement With hundreds of deadlines and millions of tasks assigned to QA every cycle or sprint, sometimes you can get a little overwhelmed and not even know where to begin to get it all done nor how to improve your skills and techniques outside those other demands. Several works of classical literature spend some time on those very topics. The Autobiography of Benjamin Franklin, written by Benjamin Franklin, tells you what a wonderful fellow Franklin was. Although self- promoting, Benjamin Franklin, the son of a candle maker, took a printer’s apprenticeship and parlayed that experience into founding a magazine, entering the political realm, and serving in many early American government positions before and after the Revolution. To accomplish this, he worked very hard and kept a rigid schedule of self-improvement. It’s harder in the 21st century to keep on task and on a tight schedule with the distractions of the Internet and mobile devices, but Franklin’s discipline can offer some guidance and inspiration. You can find a fictional flip side to Franklin’s discipline in The Great Gatsby by F. Scott Fitzgerald. The title character also rises from humble beginnings to a position of opulence through rigid self-discipline. As a Lost Generation novel, though, the story does not end as well for the characters as Benjamin Franklin’s does for him, but The Great Gatsby does feature more cloche hats. Books About Adapting In the hurly-burly world of software development, where the battle’s lost and won daily, the world shifts daily in subtle or dramatic ways. Methodologies change, underlying technologies change, and the participants keep getting younger. In this world, you have to adapt quickly to changes, and classic literature can help you in two ways. First, because literature can extend from Chaucer to the 21st century, you have to quickly adapt to different languages and different eras Continued on page 7
  • 7. 7 Follow us at www.twitter.com/testingclub Who needs questions? Lets just test! http://buff.ly/RxcOqR Br ief His t orY OF Time a Fr enchEdition Continued from page 6 just to read the books. A Jane Austen novel circa 1813, a Charles Dickens novel circa 1835, and a Thomas Hardy novel circa 1886 differ from each other in setting and in language, and they’re all quite different from the aforementioned The Great Gatsby. By varying your reading, you’ll train yourself to adjust your in-reading understanding to the work itself. This habit will transfer as you change between development projects and between problem solving for different discrete industries with their own concerns and argots. If you shift your mental gears between the country estates, dismal factories, and the wilds of India without grinding them, you should easily adjust between a client or project in the Internet retail space and a client or project writing a mobile application for courier tracking and delivery. Secondly, you can watch and perhaps learn from characters that adapt quickly. If you take Rudyard Kipling’s Kim, you’ll learn from a young man who moves between Buddhist, Moslem, Indian, and British worlds as he participates in the Great Game. As a bonus, if you grab a collection of Rudyard Kipling’s works that includes Kim with Kipling’s short fiction, you’ll develop definite agility in switching contexts, as Kipling’s characters range from Americans in England, English soldiers in India, English officers in India, naval officers at sea, a mongoose, and more. Not-So-Great Books Of course, the springboards for your creative vaults do not have to come from the Great Books. You will find ideas and inspiration in other books, too, if you’re open and attentive. You might find solace and camaraderie in the hardboiled noir books or in the second wave pulp paperbacks of the 1960s through 1980s; I certainly fancy myself a Mack Bolan of testers. Dan Brown’s puzzle-and- conspiracy filled tomes might give insight into how to approach an unfamiliar interface with an eye to reading the messages and alternative workflows that others do not see. Maybe a comic IT heist novel will satisfy your revenge fantasies with a side order of physical system analysis and problem solving. You can also glean ideas from nonfiction books, particularly in other specialties. Perhaps a book about real estate sales can illustrate interpersonal relations in a new way and help you to communicate more effectively. Old technology or forgotten management style books might include ideas that you can tweak for the present day. Biographies of military leaders provide management ideas and lessons, only some of which involve harrying the north. Regardless of what you choose to read outside the field, you’ll make greater leaps of creativity when you add unrelated content to your mind. You’ll have more material from which to draw to try new things and to see things differently from your peers. At the very least, you’ll learn new quotes to sound classically educated and Latin words to brandish like gladii. As a child I devoured the Isaac Asimov “I, Robot” series - they were wonderful stories. The world of the future had robots bound by 3 Laws of Robotics. However despite this, these machines would behave in odd and occasionally dangerous ways. Invariably the cause of this under investigation would be some unperceived complexity in a scenario or situation, which was overlooked. I think particularly the tales of Powell and Donovan spoke to me as “hero testers”. I think these stories taught me a lot how to investigate and challenge “bugs” (or rampaging robots), especially when others say that a problem in their area of code is “inconceivable” (also see The Princess Bride for use of that word). “What Do You Care What Other People Think?” by Richard Feynman is an inspiring and life-changing read. He was a physicist on the Manhattan Project, and the book talks a lot about his scientific outlook on life. A good portion of it involves his work on the investigation into the Space Shuttle Challenger disaster, and how he encountered a culture at NASA which believed too much of it’s own hype. Some notable quotes from this book include “When playing Russian roulette the fact that the first shot got off safely is little comfort for the next” and “reality must take precedence over public relations, for nature cannot be fooled”. IntheTestingPlaneteditorialteamwewerereallyimpressedwithBrian’sarticleasitechoedourownexperience.Sothe questionwentaroundourvirtualoffice“whatnon-testingbookshaveinspiredyouinyourtesting”.Hereareouranswers... Br ief His t orY OF Time a Fr enchEdition How to Use Your Eyes by James Elkins - It’s hard to capture in a few paragraphs what this book has offered my testing, as every time I reflect on it, I MIKE TALKS DAVID GREENLEES think of areas that it can be extended to. One notable mention would be the prompt throughout the book to view beyond what you’re seeing. I like to refer to it as the ‘extension of sight’. Without giving too much away, the first section of the book focuses on postage stamps. Yes, there is time spent describing the actual picture on the stamp, which is fascinating, however the background story that the picture tells is the most amazing part. This book constantly reminds you to think about that story. When I first began reading it I said to myself, “Oh no, not postage stamps... how boring”. It ended up being very interesting and very educational. Since reading this book I have remembered to look beyond what’s directly in front of me. This has helped me in many aspects, however most noteworthy would be gaining an understanding of why certain decisions are made. Instead of looking at a decision in isolation, I now seek that background story. It’s a powerful thing to find out what drives us as human beings, and all we have to do is employ an ‘extension of sight’. The Ender Series by Orson Scott Card - Not one book but a series, the story of Ender (or Andrew) Wiggin by Orson Scott Card has had a big influence over my thinking since I read it about six years ago now. It really is an outstanding piece of science fiction, but in addition to this it introduced me to the subject of anthropology. Being primarily the study of human behaviour, the author applies this science to the observation of alien races resulting in some quite fascinating story arcs. Applying this thinking to the dynamics and behaviours of project teams has been and will no doubt continue to be a fruitful source of valuable information. Wiggin goes on from the seminal Enders Game to become the first Speaker For The Dead, which entails studying the life and times of a deceased being and telling their warts and all story by way of a public speech. To quote Wikipedia on the matter - “This speech is not given in order to persuade the audience to condemn or forgive the deceased, but rather a way to understand the person as a whole, including any flaws or misdeeds.” The quote could almost be a testing mission statement - a mandate not to judge, but to provide a common understanding of the product as a whole, including any flaws or misdeeds. I like it! Culture Shock: A Handbook For 21st Century Business by Will McInnes - I’ve long struggled with how traditional businesses are run. The more I experienced working in ‘them’ the more I became curious of understanding what it takes to create a business. As I’ve built up the Software Testing Club I’ve always known it would be built lean, on a budget, but with high integrity in mind. Culture Shock is a great book that covers what many forward thinking companies are doing to approach business in a different way. Some things these companies do seem crazy, but it doesn’t stop them being successful. Lean StartUp by Eric Ries - This follows on from theme business theme behind Culture Shock. I love business and am always coming up with new ideas of what we could be doing. Some are small additions to what I’m already doing, others are inter-related... and of course there are the pipe dreams. What it has done is allow me to focus on what I’m doing, taking things one step at a time and evaluating things as I go. I haven’t been a strict ‘lean startup’ but the philosophy is very much in my heart. □ ROSIE SHERRY SIMON KNIGHT
  • 8. 8 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Are You in QA? No. Am I a Software Tester? Yes! By Michael Larsen The above statement may seem a little flip, but I have this question asked of me quite frequently. Somehow I will be involved in a discussion and something will come up about testing, and when I talk about it, a natural question to be asked is “oh, so you must be in Quality Assurance!” For years, I answered “yes”, but I don’t answer that way any longer. Now, when people comment that I must be in QA, I answer, “Am I a software tester? Yes!” For many of the people I talk to, the comment doesn’t just hang in the air; they notice the difference, and often they ask me about it? What do I have against QA? Why do I avoid saying those words, and why do I substitute the words “software testing” instead? The roots of this come from my further understanding of the dynamics and the environment(s) that make up software, how it is created, and the environment in which it is built and where it runs. The term Quality Assurance stems from the idea that there should be quality standards. software exists in an unusual state that is never 100% duplicable. If we were to take two computers, loaded with the same operating systems, and with as close as possible the same hardware, and we ran those two systems, with as close to identical configurations as possible, we still could not, with certainty, say that the machines would behave identically. We could guarantee that the metal parts were stamped out to make the case with a high amount of precision, but the sustaining of the electronic states of an operating system will be different on each computer. The vagaries and differences in the speed of ram, the speed of the hard disk, the length of the cables used inside the chassis, the power load at that given moment, and any programs installed and activated on the system will make for a different experience. This is a key reason why I have come to view software “quality assurance” as a misnomer. Metaphorically, installing and testing software is less like building a house on a concrete foundation, but more like building a houseboat that floats on a lake (and to extend the metaphor, building it on the lake instead of on solid ground). The reasons run even deeper than this however. With Quality Assurance, there is both an expectation and an understanding that we can do something about the issues we find. That’s what Assurance means. I assure that this product has a level of quality. For most of us that work in this space, that is not likely, nor is it really even feasible. To assure software quality, we would need to have the ability to make changes to the system, to “retool the line” so to speak, so that we can make sure that the system is doing exactly what it was Continued on page 9 This also stems from the factories in which items are made. Whether they are made of wood, plastic or metal, factories have a vested interest in their parts being designed to have few defects or broken pieces. The idea of quality assurance grew out of the factories and assembly lines looking to guarantee, as much as possible, the parts that were made not only fit the correct measurements and specifications, but that they also fit into the other parts to assemble a finished product. If we are talking about a drill press, and the ability to make for equally distributed holes into a piece of sheet metal, having the ability to guarantee that process works consistently is realistic. It’s a very stable process, and something that can be assured (or as close to a guarantee as is possible). The metaphor of a factory and an assembly line feels natural when we talk about software. Functions plug together, modules can be connected to make larger programs, and numerous “moving parts” can be “assembled” to make a finished product. The similarity, however, ends here. Unlike a drill press or a metal stamping machine, MichaelLarsenisaseniortesterwithSideReel.com,awhollyownedsubsidiaryofRoviCorpora- tion,currentlyworkingattheRovisiteinSanFrancisco,California.Overthepasttwodecades, hehasbeeninvolvedinsoftwaretestingforproductsrangingfromnetworkingequipmentto capacitancetouchdevicestoInternetapplications.MichaelservesasaDirectorfortheAs- sociationforSoftwareTesting(AST)andistheChairoftheEducationSpecialInterestGroup. HeactivelyteachessoftwaretestingthroughtheBlackBoxSoftwareTestingseriesofclasses offeredbyAST.HeisablackbeltintheMiagi-doSchoolofSoftwareTesting,istheco-founder andprimaryfacilitatorforWeekendTestingAmericas,andistheseniorproducer(andfrequent commentator)forSoftwareTestProfessional’s“ThisWeekinSoftwareTesting”podcast. AUTHOR PROFILE - MICHAEL LARSEN WWW.MINISTRYOFTESTING.COM CO-CREATING SMARTER TESTERS EDUCATION • COLLABORATION EVENTS
  • 9. 9 Follow us at www.twitter.com/testingclub We’ve created an ultimate resource on SBTM - http://buff.ly/QlZkO0 Br ief His t orY OF Time a Fr enchEdition Continued from page 8 specified to do. In the software-testing sphere, at least in the environments that I work in, we don’t have that control or power. We have to have a programmer make those changes. Thus, the ability to assure quality is not in our hands. The biggest reason, though, has nothing to do with these situations I’ve described. The reason I dislike using the term “Quality Assurance” the most is the false expectation it gives. It tells people that we can assure quality, in a way that says we will prevent any defects from getting to the public. While yes, that is ultimately what we want to do, the fact is that it is impossible to test completely, to examine every conceivable situation, and cover every eventuality to assure that there are no defects. If a defect does get out, we have set ourselves up for blame. We are Quality Assurance, our job is to assure the quality of the product, and therefore any bug found in the fields is our fault. Nonsense. While it is true that I, as a software tester, did not find that particular issue, neither did the programmer who wrote the module, or the product owner who accepted the module. We’re all on the same team. Setting ourselves up to be the judge, jury and executioner will merely leave us to be blamed if we miss something. These are of course all semantic arguments. The fact is, Quality Assurance is a long established term, and for the most part it is synonymous with being a tester. Still, I prefer using the words “software tester” for a variety of reasons. First, it accurately effects what we do. We test the software. Not an idealized piece of software on some mythical perfect machine, but software that flows and sways like that houseboat on a lake. We look at a product and we explore it. We discover with the resources at our disposal how the software behaves, and based on that behaviour, we explore in different places. Jon Bach has a metaphor that I personally love1 , in essence saying that instead of trying to present ourselves as the “bug shield’ or the “last tackle on the field” we should use something closer to who and what we really are. We are storytellers. We are journalists. Our goal is to explore, study and learn about a product, from as many different vantage points as we can, and then, based on those vantage points, explain what we see. Tell as complete a story as possible. Show the software team and product owners the who, what, where, when and why of the software. We are providing the development and product team, and the customers, with information about the fitness of the product. Ultimately, the product team and the programmers will determine what they will do with that information. Often it will be to take the problems that we have found and provide fixes for them. Sometimes, it will be to leave the issues as is. Each time I have this conversation, I have the chance to change someone’s mind on how they see software development and what we as testers really do. Often, I find that I cannot change opinions; the long-term use of the QA term is too strong and ingrained. But sometimes I find someone who gets what I’m saying, and they start to view software testing differently. I’ve even heard a number of software testers likewise answer in kind when they are asked if they “work in QA?” It’s going to take time to change perceptions, but if we all work together and remind people what it is we really do, we might help others understand that the term Quality Assurance belongs in the factory, while software testing belongs with the code we work with every day. □ REFERENCES 1. http://agile2010.agilealliance.org/files/Tell- ing%20Your%20Exploratory%20Story%20 Agile2010.pdf Visit www.qawizard.com to learn more and download your free 30-day trial today. Learn More QA Wizard: Intelligent Tools for Testers Automate your web, Windows, and Java testing, and gauge the real-world performance of web sites with QA Wizard Pro. • Create adaptable scripts that require less maintenance with an object-based recording engine and global control repository • Test the latest technologies including HTML 5, Java, .NET, Qt, AJAX, and more • Simulate hundreds or even thousands of users to measure the real-world performance of your web site With Resource Thief you don't need to maintain virtual machines, or create crazy configurations in the lab to test application behavior in stressed environments. • Limit disk, memory, and network access to specific applications • Run existing, or new test cases against the application under test • Use your existing diagnostic, testing, and development tools during stress tests Resource Thief Stress Testing QA Wizard Pro Automated Functional & Load Testing
  • 10. 10 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Next Rapid Software Testing course is in March with James Bach in Brighton! http://buff.ly/Qd3Py9 Br ief His t orY OF Time a Fr enchEdition A Creative Tester Asking for Trouble By Jeffrey H. Lucas I always used to get into trouble. It always started the same: I would be given a straightforward task of running a pre-defined, established test script written resulting from an analysis of the requirements that was conducted at the very beginning of the project. I would start the procedure as usual, verifying the test script appeared to address the requirement, and begin running it. Testing is not about following a script, but applying your imagination and creativity to the testing of the product. However, if applied in a static fashion, scripted procedures tend to suppress this creativity. To illustrate this, I provide two test sessions involving a fictional inventory tracking system for a merchandise distributor (“Foobar Mercantile Corporation”). The purpose of this imaginary application is to provide greater visibility to the product inventory in transit from the warehouses to the distribution locations. Some mockups are provided to help follow the testing. Session 1 – Following the script The first session involves an 18-step pass/fail “scripted” procedure. The stated purpose of this procedure is to demonstrate the requirement for a system administrator to provide reporting capabilities to groups. The procedure has been verified (i.e. run) multiple times and is expected to take about two minutes to complete as a part of the acceptance test process for this delivery. The procedure demonstrates the requirement by assigning the reporting permission to the default Domain Administrator group (who administer geographic/physical locations within the system). In this procedure, I represent my thought process using imaginary users I create in my mind to visualize the application in use. Compare the tangential, sometimes chaotic, thought lines to the simple flow of the script. ME: Are we set to run the procedure? CO-TESTER: Ok, let’s do this and then we can take lunch. I will call out the steps and you execute. “Step 1. Login as a System Administrator and … “) (Keyboard-restricted User) ... I can’t navigate out of the login area (Ref. 2) using the Tab key ... I have to use Ctrl+Tab … very difficult … I want to access the company link at the bottom... (Company President) ... Why is our cor- porate logo on the company link (Ref. 3) so blurry looking? … Very bad for our corporate image... (Keyboard-restricted User) ... I can access the page … navigating back, but the focus appears to move to the top of the browser with no indication of focus (Ref. 1)… this is getting -very- difficult to use … Me: I’ll make a note of those. Co-tester: “... add the ‘Reporting’ permis- sion to the Domain Administrator group.”) (SystemAdministrator) ... give me a break … I’ve been doing system administra- tion for 9 years and you want me to give that permission all of the Domain Admins? ... Me: So what are you saying? This function is a basic, established part of most applica- tions. A best practice that was identified as a necessary product enhancement early in the process. I read it right here. (System Administrator) ... if you knew the Domain Admins like I do, you wouldn’t be so... (DomainAdministrator)...Iresentthat... Me:Wait...soyoumustbeaDomainAdmin.... (DomainAdministrator) ... no I got moved to section management in trans- portation, but they still have me overseeing some of the warehousing administration... Me:Soyouhavemixedroles!Thatmaymake addingthe“Reporting”permissiontotheDo- mainAdministratorgroupabitofaproblem. (Ex-DomainAdministrator) ... Yup, I agree that it shouldn’t be applied unthink- ingly. I do have a need for it, so perhaps a limited application based on job description and not just titles... (AdministrativeAssistantw/de- mandingboss) ... Wait a minute; don’t forget about me ... I also need access to support planning... Me: So you think the current group permis- sion implementation may be too limiting and should be rethought? (Administrative Assistant w/ de- manding boss) ... yeah, maybe a special group assignment based on identified need rather than administrative title or formal job description... (Ex-Domain Administrator) ... that’s doable... (System Administrator) ... yeah, I could go along with that... Continued on page 11 Figure1:LoginPage
  • 11. 11 Follow us at www.twitter.com/testingclub iOS Testing MindMap - http://buff.ly/QRSh2J Br ief His t orY OF Time a Fr enchEdition Continued from page 10 Co-tester: “Step 2. The Domain Admin- istrator receives an email notifying them of the permission change with a link to the application login page.”) (SystemAdministrator) ... Whoa! That’s just asking for a spear phishing or whaling attack... (BlackHatattacker) ... Heh heh... Co-tester: OK, verified. Next: “Step 3. Login as a Domain Administrator, open the permission panel, and verify the Reporting permission state has changed from green to red.” (Colorblinduser) ... Now, exactly how am I supposed to tell that?!? (Ref. 4)... Me: Ok, hold on. I think there may be some serious issues with this user flow. I’m see- ing a lot of areas that we need to stop and explore for a bit. Co-tester: Again!?! Now why do you think that? You know they expect us to have this test suite completed before we leave today. Me: Yeah, I know. Let’s just say a little voice told me. This is just a sample of how restrictive test processes can inhibit the creative imagination of a tester. Many testers with a wealth of experience in testing have experience similar sessions in the past. The key is to incorporate the imaginative, inquisitive mind into the testing. Session 2 – Putting your imagination, not your script behind the wheel To leverage the power of a tester’s imagination, you have to give a vent to the thoughts – thus giving an actual voice to the imaginary users. The second example involves a test session with a stated charter to explore the requirement for a system administrator to provide reporting capabilities to groups. The charter was created as a result of functionality recently added to the application. It has never been run before, being a product of previous design and exploratory sessions. As such, the procedure has no fixed time limit, but 30 minutes have been set aside for this initial session. This is a paired test, with one tester running the application and the other observing, providing insights, and taking test notes as necessary. Note that the dialog between the testers has replaced the imaginary user dialog. Me: Are we set to run the procedure? Co-tester: Ok, let’s do this and then we can take lunch. Based on our surface analy- sis, we’ve identified some potential risk points in the login screen, but the permis- sion-based implementation they created should be straightforward. Me: I’m initially going to investigate the login accessibility risk. Hmmm … I can’t navigate out of the login area using the Tab key … make a note that the navigation requires the use of Ctrl+Tab. Co-tester: That company logo at the bot- tom looks kind of ugly. I’ll make a note to see if we can improve it. Me: Whoa! Things were going OK until I navigated back from the link. See the how the focus was moved to the top with no focus indication? Let switch back to mouse operation and try out the reporting assign- ment. They implemented a permission- based system based on user role. Co-tester: You know, there may be a lot of different users wanting this permission. This implementation may be too restrictive. I’ll make a note to investigate this further. Me: I’ve assigned the permission to the Do- main Admin. I’m logging in as that user Jeffrey H. Lucas is a Senior Software QA Analyst located in San Antonio, Texas, U.S.A. He has nine years of experience in software testing, and is the test lead on a small product development team. Prior to that, he has over ten years of test automation experience in other industries. His current interests are in the development of software test practices suitable for small development team environments. AUTHOR PROFILE - JEFFREY H. LUCAS and checking the permissions panel for the permission. Uh-oh, we have a problem. The indicator just uses a red-green color scheme with no alternative. That is defi- nitely an accessibility issue. Co-tester: Check your email. I configured the system to send you an email notifi- cation on change of permissions. Ouch! Do you see that link at the bottom? They should know better than to redirect to login screens through email – it can be exploited for spear phishing and whaling attacks. Me: I didn’t see that. I’m glad you were avail- able to assist with this. Software testing needs to incorporate intelligent review of oracles (such as stated user needs), risks, and user flows throughout the software development cycle. These two sessions illustrate some of the ways that the imagination and creativity of the tester can be leveraged: • Use charters to guide the test session, or specifically provide instruction that scripted procedures are guidelines only. If scripted procedures are necessary, then allow markup Continued on page 12 Figure 2: Permissions Dialog
  • 12. 12 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MindMap-TestingInProduction-http://buff.ly/PDCo23 Br ief His t orY OF Time a Fr enchEdition Continued from page 11 feedback from the test sessions that are reviewed regularly for incorporation. • Use paired testing to provide a “voice” to the thoughts of the testers. Additionally, this provides a second set of eyes that are not preoccupied with the mechanics of running the application. • Create the test sessions from previous session questions and ongoing risk analysis, instead of static procedures that are run blindly. • Use regular, short development cycles to analyze, validate, design, develop, and test the functions. The sessions should incorporate all players in constant communications instead of verifying requirements analyzed months before by people you have never met. In the case of the first session, the voices can be ignored for a while, but it comes to a point that they demand to be heard (either before or after delivery). Perhaps the better approach would be to get to know the end users of the product to champion their values and mindset by incorporating actual users (with real voices) in the process. (Chorus of voices) ... Amen to that... □ By Vipin Jain and Anubha Jain The buzzword is globalization. Shopping has taken the e-route with new ecommerce sites showing their presence each day on the World Wide Web. Credit card usage online has gone beyond unimaginable proportions. Each company has a web presence. Add multi-language support to this mix and a true globalized website is there. Wow! How easy is this or is there a catch? Web site globalization is as complex a process as it gets, and demands significant investment of time and resources. Oftentimes however the planning is poor and goals are not clearly defined; a wide range of technical hurdles and linguistic embarrassments easily stalls web site globalization. The solutions are non-trivial; however there are a number of “secrets” that may help you avoid some of the pitfalls of building and testing a multilingual site. Web globalization is still very much a practice one “learns by doing.” For many developers and businesses the thought of having multi language support within their portals might never have crossed their minds. They should however think of the benefits they will Continued on page 13 NEWSIN BRIEF Automation Samurai have just released a new QTP add-in that supports all versions of QTP. It utilises the latest Selenium technology (webdriver) to extend QTP features. With this Add-in QTP now can support the latest versions of Safari, Opera, HtmlUnit, Chrome, Firefox not only on Windows, but on all major platforms such as MAC and UNIX. http:/ /www.automationsamurai.com Testing Multilingual Websites Figure1.GoogleDriveMultilanguage www.testninjas.com
  • 13. 13 Follow us at www.twitter.com/testingclub CheckouttestingeventsfromMinistryofTesting-http://www.ministryoftesting.com/training-events/ Br ief His t orY OF Time a Fr enchEdition Continued from page 12 get from supporting multiple languages as this will likely lead to an increase in the rate of conversions, more user traffic and a wider global reach. When a multilingual site is planned and created, there is much to consider and it’s really all about localization and knowing the culture associated with the language. Although the practice of web production has matured rapidly over the years, the practice of web globalization is still very much in its infancy. In “Secrets of Web Site Globalization” John Yunker mentions that although nearly every US business now has a web presence, only 15% offer more than one language. With so few examples to build upon, the web manager planning a multilingual site is often left with little direction and support. There are many tips and things to take care of while developing a multilingual site; there are very few such tips for testing such a site. What should we as testers, do to make sure that the site displays the correct content for all languages?As an example, text in English has been embedded in the image shown here. Allwords,orkeys,arewritteninonecolumn.Thereareseparatecolumnsforeachlanguage.Each languagehasbeengivenacode,asspecifiedinISO639-2Code(http://www.loc.gov/standards/ iso639-2/php/code_list.php).WehaveusedGoogleTranslatefunctionwithintheGoogleDrive spreadsheet.ItssyntaxisGoogleTranslate(“text”,“sourcelanguage”,“targetlanguage”).Source languagehereisEnglish(en)andtargetlanguagesarementionedineachcolumn. FIGURE 1. NOTES Figure2.Google’sXMLversion Testenvironmentiskeyformultilingualtesting It is not enough to simply change the default browser language and perform tests in both the languages! Depending on how it is implemented, a web site may identify correct language for its interface from the browser language setting, the machines regional and language settings or a configuration setting in the web application. It is imperative that the web site be tested from various machines; each having different operating systems written in the particular language. The browsers and operating systems should be the ones that the stakeholders have agreed upon in initial product meetings. Translations should be correct Automated web translations always provide a good start and often translate correctly. However, nothing matches a native speaker of the language, belonging to the same region as the users. The translation she brings to the table will be the best when compared with other technological translations. It is a good idea to compare automated translations from multiple sources before using them in the test. Continued on page 14 VipinJainhasdedicatedlast10yearsofhisprofessionalcareertothesoftwarequalityand testing.CurrentlyworkingwithMetacubeSoftware,India,hedevelopedhiskeyskillsindevel- opingautomationframeworksandautomatingapplications.Withaprovenrecordofimple- mentingandrefiningtestprocessesforvariousclientsacrosstheglobe.heistheauthorof severalarticlesandsevenwellsoldbooksinIndia.Heisactivewithinthesoftwaretestingcom- munity,speakingatInternationalandnationalconferences,writingarticlesandcontributingto variousblogsandforums. AUTHOR PROFILE - VIPIN JAIN When we see the page containing this image in another language, the text on image still shows in English language. How to plan for such scenarios? After a few tips for testing multilingual sites, I will explain a simple but effective mechanism to plan for such tests. Know the cultural aspects A simple but challenging aspect of testing multi- lingual web sites is that each language might belong to a region that has a distinct culture. They may have a preference of colour (Spanish people prefer orange), text direction (people in middle east prefer right to left), format of salutations and addresses, measures, currency etc. are different in different cultures. Apart from language translation, other elements of the user interface should also be correct. Subscribe to Multilingual Compliance News
  • 14. 14 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MindMap-TestingandChecking-http://buff.ly/R127wk Br ief His t orY OF Time a Fr enchEdition Continued from page 13 Test Labels first Labels are the most static items on a web page. They are a great starting point for testing, as they tend to expand on translation. It is important to find any issues related to their truncation, overlay on/ under other controls, wrong word wrapping etc. Once this has been done, another area to focus on is error messages. The test should include generating all the error messages. If some text is not translated, three possibilities can exist. The text is missing, its English equivalent is present or junk characters show up in its place. A test model for multilingual sites Environment We have an application with support for 5 languages. The automation suite is written in Java. The test framework uses different property files for each language. These files are in XML format to be used by Java code. Challenge The challenge is to obtain the translation, maintain the translated text and generate correct property files in the form of XML. We are not going to purchase any expensive tools, but have used tools like Google spreadsheets and a couple of spreadsheet functions to achieve the goal. Just by the use of few simple spreadsheet tricks, we were able to generate properties files to be used with our automation suite. Manual translators are not available; hence we use Google translation service, which is the next best thing to manual translations. AnubhaJainisworkingasAssociateProfessor&Head,ITdepartment,InternationalCollege forGirls(ICG),locatedinJaipur,India.Anacademicianforlast11years,shehasbeeninvolved inteachingandmentoringseveralstudentsinthefieldofComputerScience.Knowingthat teachingisthebestformofgivingknowledgebacktosociety,sheworkedasalecturerin Subodhcollege,JaipurbeforesettlinginhercurrentroleatICG.Sheiscurrentlypursuingher PhD,infieldofInformationarchitecture.Anubhaistheauthorof9booksandaregularcontrib- utorinvariousforums. AUTHOR PROFILE - ANUBHA JAIN REFERENCES • http://www.loc.gov/standards/iso639-2/php/ code_list.php • http://support.google.com/docs/bin/answer. py?hl=en-GB&answer=1388877 Figure 1. shows what a Google Drive spreadsheet looked like. See figure 1. notes. Once this table is ready, our next step is to generate valid XML files, to be used by our automation code. (see figure 2.) This then needs to be copied for each language and once it is, all of the required XML will be ready. The automation script now needs to be tweaked. The code needs to identify the control named “Advanced Search” and click on it. Whatever language you are on, the control’s internal name shall remain as it is, as shown in the XML. Hence, our script shall run fine on all pages; irrespective of what language user has chosen. So with a couple of Google functions, and some basic coding skills, we can develop the data sets for each language and the framework developed to run our automation scripts is quick to recognize it. This will save all of us a lot of time in writing separate scripts for each language page. □ Using simple concatenation formula, we develop the XML within few seconds. =”<name=”&$B$4&”>”&C4&”</name>” Final XML looks like <?xml version=”1.0” encoding=”ISO-8859-1”?> <key-value> <name=Hello>¡Hola</name> <name=Advanced Search>Búsqueda Avan- zada</name> <name=ISBN List>ISBN Lista</name> <name=Author List>Author</name> <name=Title List>Lista de títulos</name> <name=New & Used Textbooks>Libros Nuevos y Usados</name> <name=Bestselling Textbooks>Libros más vendidos</name> <name=Bestselling Authors>Autores más vendidos</name> <name=Bestselling Titles>Los títulos más vendidos</name> <name=Used Books>Libros Usados</ name><name=ISBN Search>ISBN Búsque- da</name> </key-value> AUTOMATION CODE a community for software testers WWW.SOFTWARETESTINGCLUB.COM
  • 15. 15 Follow us at www.twitter.com/testingclub BugMagic-http://buff.ly/Q9FrPM Br ief His t orY OF Time a Fr enchEdition Figure1:JUnitOutput By Simon Knight It doesn’t seem like so long ago that I took to the Internet1 and vented about being frustrated with the limitations of the technologies we’d chosen for a big test project. Being a forward thinking kind of team, we’d decided to go ahead and utilise what was then the cutting edge behaviour driven design framework Cucumber, to help achieve our goal of rapidly identifying functional errors and regression issues using test automation, while building and deploying the software using the Hudson continuous integration environment, latterly Jenkins2 . Cucumber is a tool that can be used to support behaviour driven development using plain text feature files, which are then automated using Ruby. Features consist of scenarios, which in turn are made up of steps. Feature files and their corresponding scenarios are turned into automated tests by adding an additional layer of Ruby code, defining for each step what specific actions should be taken against the code under test – step definitions. Unfortunately it became quite evident from the early stages of the project that things weren’t really going to work out how any of us wanted. The original plan was for the intrepid tester (yours truly) to handle the test cases, or features in Cucumber speak, and then pass them along to the developers who would write the lower level implementations required to hook into the application code. This never really worked out how any of us had hoped or imagined it would though… The team of developers had their work more than adequately cut out for them in trying to deliver the necessary code to get the application up and running in the first instance. Implementing my test features was way down their list of to- dos, and with a full time exploratory testing gig, I simply didn’t have the time or knowledge of the underlying production code to be able to implement the features (lower level [Ruby] step implementations that hook into the production code) myself. Gradually the number of features left unimplemented grew, leaving the team feeling as though valuable time and effort had been wasted. Cucumber feature files without the corresponding step-implementations translated into considerable technical debt. Ian – technical architect for the project recalls – “There was a lot of frustration about tests SubSteps having been written but not implemented by the development team. It seemed like you in particular banged on about tests not being implemented for weeks on end - every single stand up meeting! Mixed feelings overall. The concept was good but the execution was very flawed.” We meet up in a pub down the road a little ways from our Sheffield office. Ultimately I wanted to talk to him about how we went from this place of supreme frustration for all concerned, to developing an alternative, ATDD (Acceptance Test Driven Design) style tool that met the needs of both testers and developers. BDD/ATDD Behaviour Driven Development (BDD) grew out of advances in Test Driven Development (TDD) and in particular the idea that unit level tests could be expressed in natural language rather than method declarations3 . Whereas BDD tests are likely to be owned (created & maintained) by the developer implementing the code under test, Acceptance Tests are more likely to be owned by the tester/subject matter expert. Jennitta Andrea defined ATDD as “the practice of expressing functional story requirements as concrete examples or expectations prior to story development. During story development… The story is “done”—deemed ready for exploratory and other testing—when these scope-defining automated checks pass.”4 Over mouthfuls of steak pie and real northern ale, he goes on to talk about the situation from his perspective: “The amount of time and energy required from the developers to code step implementations once tests had been scripted caused a lot of problems. It also seemed like there was a lot of repetition in the step implementation code - redundancy. I felt that it might be possible to expose the functionality in a more tester friendly fashion and move this work out of the developer domain thereby making it more manageable.” Did the other developers share your frustration? Ian - “To a degree. I’m not sure if I was being especially obstinate but I just wasn’t happy. I felt like it was slow and that it could be a lot of easier. “We were using Cuke For Duke originally which is a Ruby gem that bridges the gap between Java and Ruby, enabling you to write your step implementations in Java. The calls need to pass through the JRuby [version of Ruby that runs in the JVM] stack, into Ruby and then back out again ultimately into Java which is quite convoluted and a bit of a nightmare to debug - lots of information is lost. This may have changed now that Cucumber can be run as pure Java5 . “I was struggling to run Cucumber locally, in debug mode. I was using the original Ruby based implementation of Cucumber and to try and resolve the problems I was having I wrote a little Java application to run the steps for me. I realised at this point that there wasn’t actually a great deal of effort involved in converting the steps to Java instead of Ruby, and basically decided to keep going.” Did you experience any other problems? Ian - “My own coding experience was a bit of a limitation. If I’d studied computer science at university I might have had the opportunity Continued on page 18 SimonKnightisatesterbydayandEditorofTheTestingPlanetbynight.Organiserofocca- sionalBrummieTesterMeetups,Simonhasalsobeenknowntospeakandwriteonoccasion. Readhisblogherehttp://sjpknight.comorfollowhimonTwitter@sjpknight. AUTHOR PROFILE - SIMON KNIGHT
  • 18. 18 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag MySurfaceChallenge-http://buff.ly/Q9FwTJ Br ief His t orY OF Time a Fr enchEdition Continued from page 15 to develop a compiler, which would have saved me some grief with the early SubSteps implementations. At some stage fairly early on I’d managed to code myself into a corner and had to rewrite the entire thing pretty much.“ Rewind back to the last few weeks of 2011. The project on which Cucumber had caused so much tension has been brought back from the dead and, like a kind of Frankenstein’s- monster-of-test; we decide to try to and splice both tools together. We end up with a set of legacy Cucumber features, and the beginnings of a new pack of tests – scripted, and implemented using the new tool and mostly without developer intervention. Both feature files and their step definitions can now: • Be specified and developed by the testers, • Be checked in and executed during code builds without consuming lots of additional development time. Leaving developers to focus on writing code, and the test team to focus on building and maintaining the automation library and providing additional feedback via exploratory tests. Paul - the Testing Manager - relays his excitement when he realised the power that had been bestowed upon him and his testers: “We actually started working on this a couple of years back while experiencing production issues with the Cucumber tests we were attempting to use on some of our products. Being able to script tests in the Gherkin Given/When/Then format was all well and good, but we still needed technical resources to implement the step definitions and turn them into executable tests. The development resources were never available and it didn’t make sense for our testers to become proficient in the underlying code as we use various languages depending on the project requirements. Gherkin is a Cucumber specific implementation of the ubiquitous language concept proposed by Dan North in his 2007 article, What’s in a Story?6 Features written in Gherkin steps normally follow the Given, When, Then format in order to define a set of acceptance criteria. “It took a little while and a couple of false starts for the project to come to fruition, but seeing my team start to use and experiment with our new tool has been an amazing experience.” So what kind of thing is it being used for? Paul - “One of the main benefits we’re seeing is vastly increased confidence in the builds that are being delivered to our testers. With 80%+ code [lines] coverage on one project and similar results on most others, regression errors are largely a thing of the past. At least in the QA builds they are anyway; with all of our tests running against the code every time a developer checks-in, problems are normally identified long before we get around to deploying a release.” Reclining into his 3rd floor meeting room chair, Paul is in the zone: “It’s a really efficient tool to use. Every member of my team can write automated tests that cover large chunks of complex functionality in just a few lines of code. Where it really shines though is in the maintainability of the test scripts. For example on one project we have a script that performs the following actions: Given I register a user And I login as an admin user And I activate the registered user And I logout as the admin user And I instead login as the registered user When I create a profile Then I can also create an organisation profile “An element was updated on one of the pages we hit during this test so we obviously needed to update the automation code to reflect the change. Because of the way in which our tests are written, the same action can be performed in many tests by utilising the same scenario step. This also means that for my change, I only actually needed to change 3 lines of code (the SubStep definition) in order to keep the test running. This change propagates out and is picked up by 12 additional tests that utilised the same scenario step (the call to the step definition). Total expenditure – around 15 mins to fix 13 tests; that’s not bad value for money in my book!” Since SubSteps’ inception, a decision has been made to open-source it so that it can be used and extended/improved upon by anyone that wants it. Although I’ve been using it for some time to build our project test capability, I’ve long since forgotten what was involved in getting it up and running, so I decide to give it another try. I navigate to GitHub and take a look at the SubSteps repository7 . Following the ReadMe file leads me to the project documentation and an example project. I download this onto my laptop (which already has Eclipse8 and Maven9 installed – you’ll need these if you try the same thing). Once I’ve unpacked and built the project using Maven I can import it into Eclipse and take a look. Right clicking on the FeatureFileRunner.java file and running it as a JUnit test opens up a Firefox window within which I can see the fairly extensive sequence of scenario steps being executed (these will also have been seen during the Maven project build). Within a matter of seconds, the example project selftest demo is complete. SubSteps is ready to roll. Let’s try scripting an actual test... I’ll need to create a new .feature file in the features folder – we’ll call it stc.feature and inside it I’ll create a test; a rudimentary check to see that when we login to softwaretestingclub. com, a welcome message is displayed. We’ll write the high-level outline in the standard ATDD/BDD Gherkin style. Tags: @all Feature: my first test Scenario Outline: A scenario to check I can login to the Software Testing Club website Given I am on the STC home page When I login with my email <email> and pass- word <password> Then I see a welcome message Examples: |email |password| |user@email.com|password| Done. But I also need to create a lower level .substeps implementation file. I’ll call it stc.substeps to keep things nice and simple. Inside it we need to define the steps that are actually taken to achieve Continued on page 19 Figure2:GreenTest
  • 19. 19 Follow us at www.twitter.com/testingclub Jobs!TestingJobs!Findthem.Postthem.http://jobs.softwaretestingclub.com Br ief His t orY OF Time a Fr enchEdition Continued from page 18 each line of the feature. As you might expect though; things don’t work out exactly perfect on the first go (see figure 1.) I run my test for the first time… A Firefox instance opens. Elements and pages are traversed onscreen. It closes. A fail. By the looks of the JUnit output, our first couple of steps passed, but the final step failed – the page assertion appears to be incorrect. However I can’t easily identify what happened because the browser window closed after the test and everything happened too fast to monitor. Let’s change that by opening the localhost. properties file and setting the visual.webdriver. close.on.fail=true flag to false. Now if we run it again, we can see that actually the test failed on the 2nd step - ClickById xg_tab_profile which doesn’t seem to get as far as opening up the next page because the click fails to register against the correct element. I pay a visit to the documentation via GitHub to try and resolve this. I can see that there are already quite a number of bespoke commands available for me to pick up and start using immediately. If there wasn’t of course, being an open source tool I could just write one myself. On this occasion I won’t need to though, since the FindByTagAndAttributes tag=”<tag>” attributes=[<attributeString>] command looks like it will do the job. Eventually I end up with the implementation below (also see figure 2.): Define: Given I am on the STC home page NavigateTo http://www.softwaretestingclub. com/ AssertPageTitle is “Software Testing Club - An Online Software Testing Community” Define: When I login with my email <email> and password <password> ClickLink “Sign In” FindById signin_email ClearAndSendKeys “<email>” FindById signin_password ClearAndSendKeys “<password>” FindByXpath //div[2]/div/div/div/div/div/ form/div/fieldset/dl[3]/dd/input Click Define: Then I see a welcome message AssertPageTitle is “Software Testing Club - An Online Software Testing Community” FindById welcome-text And… A green test! Clearly in a real-life scenario we’d want to expand upon this considerably and add a layer of manual exploratory testing, but it’s a great start. Over another drink, Ian shares big plans for his protégé: Where do you see the future of SubSteps? Ian - “We want to expand the number of features that SubSteps supports - enhancing the language, more selectors, a greater range of assertions. We’d like to be able to check and manipulate values in the database. We’d also like to be able to do more stuff with email. The [SubSteps] Eclipse plugin10 could do with some better editing functionality, the ability to run SubSteps tests more easily, better highlighting and navigation - stuff like that. It’d be good to integrate the tool with a reporting server so that we can get more information out and track things like what features are being run over time with pass and fail stats etc. “We’ve also considered the possibility of using SubSteps as a means for driving performance tests. We need to do some more research into this though as there is a feeling that there might be some limitations with the WebDriver API. Maybe we can swap in an alternative, more suitable API for achieving this at some point... We’ve looked at the possibility of using test-nodes that can run features with results being collated into a central repository - Selenium Grid style.” So what’s changed since my rant against Cucumber and BDD style testing last year? The main upside to using SubSteps for me is immediate feedback; my biggest gripe originallyi being the disconnect between delivering test cases and seeing them actually executed against the system under test. Over time, I’ve also experienced first-hand the confidence-gain that having a large and easily maintainable pack of rapidly executed BDD style tests can bring. Over the coming months we’ll be exploring ways in which the SubSteps platform can be leveraged to enhance our test automation stack still further. If you want to, you can get involved too by downloading, using and extending the code - here: http://technophobia.github.com/substeps-webdriver/ □ REFERENCES 1. http://sjpknight.com/birmingham-software- testing-club-meetup-15112011-simon-knight/ 2. See Jerry Schwartz’s excellent Jenkins article Continuous Integration for Testers last issue, TTP8 - July 2012. 3. http://dannorth.net/introducing-bdd/ 4. http://www.stickyminds.com/Media/eNews- Letters/Iterations 5. http://aslakhellesoy.com/post/20006051268/ cucumber-jvm-1-0-0 6. http://dannorth.net/whats-in-a-story/ 7. https://github.com/technophobia/substeps- webdriver 8. http://www.eclipse.org/ 9. http://maven.apache.org/ 10. Not yet publically available. www.testninjas.com
  • 20. 20 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Areyoupartofthefacelessmasses?http://buff.ly/Q9FDhU Br ief His t orY OF Time a Fr enchEdition Testing Smartphone Apps By Ulf Eriksson For many people, the use of apps has become an integral part of everyday life, but what is it that makes an app worth talking about? Why do some apps become world news, while others are never used? First of all, let’s decide what we’re talking about here. A mobile app is a piece of software that you download, install and run on your phone, much like software on your computer. A mobile web app, on the other hand, is an Internet-enabled app with specific functionality for mobile devices and which is accessed through the mobile device’s web browser, meaning that unlike mobile apps, they don’t need to be downloaded and installed on the device. Currently there are over a million apps1 and the number grows every day. There is therefore great demand for being able to test apps; but what should you think when you’re asked to provide quality related information on an app? Should you work in a similar fashion to testing any other software application, such as desktop software? Read to find out. Purpose is the key word When you are testing an app, it is most important to have an understanding of the purpose of the app. Perhaps it is even more important than when testing traditional programs and systems. You as a tester need to know what the app is supposed to be used for and what its target audience is. There are apps that are mainly functional; that is facilitating a function in everyday life. Let us take XE Currency as an example. It is an app that tells you the value of a chosen currency against the value of a number of others. Simple. The purpose of the app is to make life easier for travellers or indeed anyone who needs The target group also has a great importance for testing. If a bank is to develop an app, security will be a top priority because it relates to users’ money. An app that can find new music, however, needs to put more effort into appearance and user- friendly search functions, given that the risk of the user losing money through the app is very low. Therefore, it is important to prioritize requirements and tests based on the purpose of the app. Apps with clear requirements are the winners It’s not news that well-formed requirements are a prerequisite for successful testing. When it comes to apps, it is not uncommon for management to state that the app should have the same functions as the company’s website. In these situations, it is important that you dare to ask questions and state your requirements. It is crucial to get decision makers to understand that an app can not, and even should not, do everything that is done on the website. That’s what the web site is for. An app needs clear boundaries on what it is to do and what it is not to do. If it doesn’t have these boundaries well and clearly defined, it runs the risk of being too broad in scope and as a consequence, poor in functionality. An app should be limited to a few specialised functions and perform them well. As mentioned above, a well-defined target group is very important when testing apps. It is therefore very effective to use personas (descriptions of the proposed end-users) from the start of the project. This is so that clients, requirements managers, and testers have the same picture of the target groups the app is going to aim Continued on page 21 a quick and efficient look at the values of a bunch of currencies. To compare it with something completely different, let’s take Flick Kick Football: a game in which you flick across the screen to shoot and curve a football into the goal. The app is pure entertainment through a fun and different idea. The app does not need to fill any particular practical purpose. In this case, the look and feel plays a much greater role. In other words, the test strategy for these two apps differs sharply because of their purpose. In the case of XE it is important to verify the synchronization of information from XE’s currency exchange figures providers, while Flick Kick Football requires no connection to external systems, just seamless game play. AUTHORPROFILE-ULFERIKSSON UlfErikssonheadsReQtest;anonline bug-trackingplatformbasedinStockholm, SwedenandistheculminationofUlf’sde- cadesofworkindevelopmentandtesting. ReQtestisahandyandsimpletooltotrack bugs,listrequirementsand manageallcommunication byanyoneinvolvedinany project.Ulfrecentlyjotted histhoughtsontesting webappsusingmobile deviceshere-http:// www.reqtest.com/blog/ testing-a-web-app-us- ing-mobile-devices/ Figure1.XECurrency
  • 21. 21 Follow us at www.twitter.com/testingclub CarnivalforaTester-RememberingOla-http://buff.ly/RxfzIE Br ief His t orY OF Time a Fr enchEdition Continued from page 20 at. This makes it possible to adapt requirements and testing accordingly. One thing that is often forgotten when specifying requirements for apps is that underlying systems might also be affected by the app. A common example of this is that a popular app generates a lot of load and data traffic for existing systems. All of a sudden back-end systems might not be able to process the traffic, which in turn might negatively impact the business. Lesson - do not forget to specify requirements for the underlying systems. Think simplicity From a user’s point of view, an app has a lot to live up to. The app must be easy to use, since hard to use apps tend to be uninstalled immediately; after all a dull app is a dead app. As a tester, it is your duty to question design and functionality. The most popular apps usually have a very simple design and ensure that users immediately understand what to do to get started. Owing to this, usability testing is very important and has to be performed and prioritized when creating an app. Make it good The next important point is stability - an app that often hangs is not going to be accepted by users. Therefore it makes sense to carry out endurance tests and to verify that the app functions together with other programs. Furthermore, response time is very important. The app must not draw too much data traffic meaning that tests should be performed to ensure that the app only carries as much data that it needs, and only when it needs it. Performance testing at the server side should be carried out to verify that the performance is acceptable should the app rapidly become popular and garner many users. Test this in advance since popular apps often spread very, very quickly. We’ve all heard about apps which went from a handful of users to 100 000 in a couple of days2 . Last but not least, it should be easy to find the app. Because there is an incredible amount of apps you need good keywords. This is something that is easily forgotten, but that should be thought of when stating the requirements and then verified when testing the app, so that the app is found by the keywords specified in the requirements. The above might seem like a tall order, but the biggest challenge is to gain users’ trust and get good reviews in the app stores. Testers can assist in delivering an app with highest possible quality, but then it’s up to the user to decide if the particular app is useful to him or her. The mobile test environment A big difference in testing an app when compared to desktop applications is that the phone or tablet itself is your test environment. Normally there are emulators in the development environment that can verify much of the app, but just because it works in the emulator doesn’t mean it will work when the app is installed on a device. There is a flabbergasting variety of brands, models with different screen resolutions and versions of operating systems on phones and tablets. Once again, the requirements play a decisive role here. We need to know exactly which models, versions and operating system functions the app will support. You can also decide on a set of core functionality that must work on a huge variety of phones and then test on as many phones and devices as you can to ensure that your core functionality is seamless between devices. Incidentally, this is the way the BBC undertake their testing3 , and they call it their ‘core experience’4 . Furthermore, some apps interact with other features and applications in your phone, which will need to be verified. There may be features for Internet, GPS, messaging, call history and synchronisation functions. It is also likely there will be new versions of the app, so it is very important to perform upgrade tests. Ensure that saved settings and data do not disappear when a new version is installed. When the same functionality is to be tested many times on different units, it is natural to start thinking about test automation. To test apps automated via emulators is rarely a problem, but just as in manual testing, difficulties begin when you have to test against the real devices. The solution to be able to automate to the physical devices is to use some form of ‘device manager’ that the automation tool can communicate with. The device manager in turn communicates with an agent that is installed on the phone/tablet. That way, you can use the automation tools you are accustomed to, even when testing apps. Be aware of the risks Apps represent a relatively new technology that is constantly evolving and therefore there is very little documentation on how to test apps, which in turn may pose a risk. It is alarming how little we know about the apps we install on our phones or tablets. It may be clear when you install an app that it requires access to your messages, phone calls and personal information, but despite that, it is installed without hesitation. Safety testing has shown that 8% of the 10 000 most used Android apps “leak” information to unknown servers5 without your consent or knowledge and similar figures have been bandied about iOS. It is therefore very important to perform safety tests in order to minimize the risk that the user is exposed to attacks. Unfortunately, the majority of apps that are available are not well tested, which means that users will have to rely on what other people have said about the app. Summary We who work with quality assurance know just how important clear requirements are. Testing an app without a good specification is like building a house without a blueprint. If on top of that you don’t even know why or whom you’re building the house for, it will rarely end happily. Therefore, the following bullets are useful to accomplish successful testing: • The purpose, what should the app do and who is the target audience? • The requirements, they must be testable. • User-friendly, “less is more” when it comes to app design. • The test environment, do not forget to verify the app on a number of models and operating system versions. • Safety testing must be performed to protect the privacy of users. What makes an app worth talking about? I would say that an app that is stable and provides benefits in a simple, good or fun way is a given success. A proper mix of usability, design and functionality makes the users choose your app and spread the word. That in itself is the highest rating an app can get. □ REFERENCES 1. http://www.nytimes.com/2011/12/12/technol- ogy/one-million-apps-and-counting.html?_r=0 2. http://www.bbc.com/news/technolo- gy-17624172 3. http://mobiletestingfordummies.tumblr.com/ post/20056227958/testing 4. http://blog.responsivenews.co.uk/ post/18948466399/cutting-the-mustard 5. http://www.itworld.com/security/185485/study- shows-8-android-apps-leak-private-data- purpose Figure2.FlickKickFootball
  • 22. 22 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag Atributetoagreatdearfriend-Ola-http://buff.ly/Q9FLhA Br ief His t orY OF Time a Fr enchEdition From Test to Driven to Design By Markus Gärtner The following could be a true story, yet it is completely invented. All characters and events in this article are entirely fictional. Phase 0 – QA does all the testing for us Tim Tester couldn’t await his new position at Big Data Corp. Inc. He had heard lots of rumours. Unfortunately his first day turned out to be a mess. He found several showstopper bugs; most of them could have been caught by lower level unit tests. When Tim walked over to Paul Programmer, he was told that the programmers at Big Data Corp. Inc. didn’t do any testing on their own. “It doesn’t say ‘unit testing’here.” Paul referred to his working contract. That’s why they have a separate QA department in the end. Tim knew that he had lots of work to do around here. When you are in the initial phase of the TDD adoption model, you write programs, and you don’t test them on their own. You rely on outside feedback about whether or not your program is working at all. This is often accompanied with hard to change code, suffering from accidentally introduced bugs, and being afraid to make any changes to the code. Often the code base is a big ball of mud, despite the best intentions to build a really good system. Testers are working overtime, project managers complain about the bad quality of the products, and during one-month customers report more bugs than the company can fix and deliver in the same time period. Eventually upper management figures out the problem, and fixes it, or maintains this vicious cycle by asking for more overtime or more staff. Phase 1 – Test automation Three months later Tim had convinced Paul to pair up with him. Tim explained to Paul how to write unit tests. “Every good unit test has three to four phases. Most often you will find some kind of Setup, then you execute some function on your tested object, and then you verify post- conditions on that object. When you involve database handling or other subsystems, you might also do a Teardown of your test. Since the former three phases are used in almost every unit test, people often refer to it as the Arrange-Act- Assert pattern.”1 Tim describes the basics of unit testing to Paul, including the different formats, assertions, how to write good assertions, and how mocking works. After Tim and Paul finished the work for the day, Paul felt confident enough to test his own code in future. Tim and Paul even found a multitude of bugs in code fragments that hadn’t been touched in a while. Up until today Paul had felt confident to use those code fragments and functions. Now he was more cautious. In the first phase towards test-driven development (TDD) programmers need to be able to write unit tests – good unit tests. Since any learning topic takes time to master, the initial unit tests will probably be bad, too descriptive, with obscure tests, test code duplication, and maybe they will even be fragile. That’s a good start, and you will probably learn something from it. At least you should strive to. Over time you will find out how to write good tests, and how to work around your code to be able to unit test it. But that’s not where you should stop, though. Phase 2 – Write the test first “Hi Paul, I just came back from a conference where folks talked a lot about writing the unit tests first, and then the code.” Tim was eager to share his newest insights with Paul. “Alright, so I write first a failing unit test, then I implement just enough to make that test pass, then do some refactoring, and start over? This sounds weird, let’s try it.” Paul and Tim worked all day on that new feature together, writing a failing unit test first, making the test pass by writing some tiny piece of production code, and removing all the duplications with the support of the passing unit tests in place. At the end of the day, Paul was unsure what to think about this new way of working. On the one hand, he saw benefits at places in the code where it was easy to drive the application. But what about GUI code? How would he write a failing test for the position of that button? And after all, didn’t this entire unit testing first thing lead to small but useless functions? Paul was unsure about the next steps, but his manager with the support from Tim kept on pushing. In the end, the quality of the product had improved greatly over the course of the past six months since Tim joined. When you have learned the basics of unit testing, you can start with writing a failing unit test before you actually write any code. At this point the move will likely feel uncomfortable, sometimes even painful. That’s ok. It’s just a sign that your learning journey has started. Keep on pushing forward. Over time you will gain enough expertise to eventually reach the next phase: Driving your development by writing a failing unit test. Phase 3 – Drive the Design Tim sneaked into Paul’s office. “What are you doing?” “I’m reading some of these online tutorials on TDD that you sent me. They explain that unit testing is not the point of TDD, but to drive the design of the application.” “What do you think about that?” “I think they have a point, but I don’t know how to do it.” “Let’s work on this together.” Paul and Tim worked on some more production code. Tim was able to show Paul more on design patterns, dependency injection and the SOLID Continued on page 23 NEWSIN BRIEF SeapineSoftwarehave launched QAWizard.com,a websitededicatedtoSeapine’s newQAWizardbusinessunit.QAWiz- ardoffers toolstohelpdevelopersand testersworkmoreefficientlyanddeliver workingsoftwaretotheir customers morefrequently. http://www.qawizard.com
  • 23. 23 Follow us at www.twitter.com/testingclub BecomeaTestNinja-www.testninjas.com Br ief His t orY OF Time a Fr enchEdition Continued from page 22 principles for good code. Paul did not understand everything at first, but at this point he had become used to learning new things whenever he worked together with Tim at the same screen. This time Paul learned how to drive his design with test-first development as well as how and when to separate his classes when they do too much work. Paul knew that it would take some more time to incorporate these lessons and be able to really work in a test-driven manner. After some time working with test-first development, you will realize there are good and bad ways to get started with TDD. You will also realize how to drive your code over several layers, and how to really use the TDD approach in order to help you develop your code in the long term. Dependency Injection2 , Coupling and Cohesion3 will become terms that you use on a daily basis. You will also know how to change the design of your application from one pattern4 to another5 , and how to use the support of your test suite naturally as an aid when making changes to the existing code base. Maybe a future generation of programmers will be able to work naturally from a failing unit test to the production code. Until then, we will have to remind ourselves to write a failing unit test for each piece of logic that we produce. But working test-driven is not the last step of the journey. There is room for still further improvement before we reach the goal. Phase 4 – Drive the Application outside-in “Say, Tim, I have a problem with that new functionality, and I don’t know how to fix it.” “Hey Paul – what’s the issue?” “You remember that specification workshop from two weeks ago? I am implementing that functionality for the table that we defined back then. It seems impossible. Where should I start?” “Ah, I see. Let’s start with the first example as failing acceptance test, and drive the application logic from that acceptance test.” “Can you show me?” Paul and Tim worked piece by piece from the acceptance test to the application layer. With each new step they reconsidered the growing structure for their support code, and tried to find out the code structure that was trying to escape the test code and become part of the production logic. Of course, each time they pulled some functionality out from the test code to the production logic, they made sure to either re-write the production code using TDD or to extract the existing structure, and retrofit enough unit tests to it. TDD is merely a stepping-stone towards driving your application from the outside in. All the other techniques that you learned in phases 1 through 3 are in place to support your critical thinking to drive the whole application from some high-level acceptance criteria. BDD enthusiasts might already find they apply phase 4; the rest of us need to go beyond what we have already learned, and learn how to extract application logic from test code, drive the application architecture and design from high-level examples, and keep them flexible to grow alongside with the application6 . Phase 5 – You don’t need code and tests anymore “Hi Tim, look, I found this article on model- based testing. They describe a framework that allows you to build the code and the tests from the same model. It’s open-source, and we won’t need to do that TDD stuff again with it.” “It’s open-source? Did you check whether they have unit tests in place? Oh, and did you find out how to test your model?” When you enter this phase, welcome to the future. You won’t need to write code and tests anymore, since there are tools for you to do that. But at this time your biggest problem probably is not writing code or tests, but the Y10k or the heavy flying car traffic around your space station. Of course this phase is sort of indistinguishable from phase 0, where you or your programmers didn’t write unit tests at all. After all, the least you should do, is to check whether the tools you are going to use match your expectations of 21st century coding practices – like including a set of automated unit tests. Of course, by looking at them, you could also spot whether these are phase 1 or phase 4 unit tests. Disclaimer The model that I went through in this article is by no means complete or intended to be a TDDMMi model, or something like that. It merely reflects my experiences with different workplaces, different programmers, and the whole idea for this article originated in a discussion back at ALE 2012 in Barcelona, Spain. You might have noticed that phase 5 is merely science fiction. After all, I would expect teams adopting TDD to take between one and three years to go through the first three phases. And these teams would produce leading edge code during that time. Please don’t rush through it, and like any model, don’t use it dogmatically. □ MarkusGärtnerworksasatestingprogrammer,trainer,coach,andconsultantwithit-agile GmbH,Hamburg,Germany.Markus,authorofATDDbyExample–APracticalGuidetoAc- ceptanceTest-DrivenDevelopment,astudentoftheworkofJerryWeinberg,foundedthe GermanAgileTestingandExploratoryworkshopin2011.Heisablack-beltinstructorinthe Miagi-DoschoolofSoftwareTestingandcontributestotheSoftwerkskammer,theGermany SoftwareCraftsmanshipmovement.MarkusregularlypresentsatAgileandtestingconfer- encesallovertheglobe,aswellasdedicatinghimselftowritingabouttesting,foremostinan Agilecontext.Hemaintainsapersonalblogathttp://www.shino.de/blog.HeteachesATDDand context-driventestingtocustomersintheAgileworld.HehastaughtATDDtotesterswitha non-technicalbackground,andhehastest-infectedprogrammersinseveraldomains. AUTHOR PROFILE - MARKUS GÄRTNER REFERENCES 1. TheArrange-Act-AssertPattern,http://c2.com/ cgi/wiki?ArrangeActAssert 2. MartinFowler,InversionofControlandtheDe- pendencyInjectionPattern,2004,http://martin- fowler.com/articles/injection.html 3. RobertC.Martin,AgileSoftwareDevelopment –Principles,Patterns,andPractices,Addison- WesleyLongman,2002 4. ErichGamma,RichardHelm,RalphJohnson, JohnVlissides,DesignPatterns–Elementsof ReusableObject-OrientedDesign,Addison- WesleyLongman,1994 5. JoshuaKerievsky,RefactoringtoPatterns, Addison-WesleyLongman,2004 6. MarkusGärtner,ATDDbyExample–APractical GuidetoAcceptanceTest-DrivenDevelopment, Addison-WesleyLongman,2012 BUGSIN THEWILD “Forthebenefitofpassengersusing AppleiOS6,localareamapsare availablefromthebookingoffice”  http://bit.ly/ios6tubemap -- Wrongturn:Apple’sbuggyiOS6maps leadtowidespreadcomplaints  http://bit.ly/anotherios6mapstory -- MistakewithStorify’sdatabase knocksdownstories  http://bit.ly/storifymistake
  • 24. 24 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag WhatisitliketobeaMozillian;Myfirsttwoyears.-http://buff.ly/RxfLYl The Evil Tester Question Time Provocative advice for testers who don’t know what to do! Listen to me at your peril IhavealwaystriedtodoExploratoryTest- ing,andfeltthatthatmademespecial. NowIhaveheardsomeonesaythatall testingisexploratory.AmIreallynotspe- cialatall,afterall? Q1. FROM Søren Harder Dear Søren, Ah, yes. It is hard when life slams our face into the pavement of reality. If you want to remain special, like me, then you have to stop listening to other people, like me. I like to use child logic on arguments that I want to treat as specious. Allow me to help you. I suspect that the definition of Exploratory Testing they use includes the null state. Then, even when the testing in question involves no exploration, it still involves exploration - it involves null exploration. You simply need to use a new, and more complicated scale, one designed to allow you to hang on to your delusion of specialness. I like to call this the ‘better’, or ‘best’, scale. Immediately discard the notion of null exploration as valid. You need to remove that if you want to be judgmental. I suggest initially setting your “fully exploratory” scale at 20%. Where 20% of testing is exploratory and 80% is not. Immediately you have become special again. And you have invoked the 80-20 rule, allowing you to make pretence of scientific categorisation. Then it becomes a simple matter of adding more grades in your scale to help you further label the testing of other people. There is one special step that I like to take. Enumerate everything that you do, and only you do, and then define ‘true’ Exploratory Testing as the specific combination of items that you enumerated. Then you not only cease ‘trying’ to do Exploratory Testing, you become the only person doing it. And that will make you really special. Helping peel faces off pavements since 2011, Auntie Evil Doyouhaveanytipsontakingthe<insert nominatedcertificationboardnamehere> certificationexam? Q2. FROM ANON Dear Anon, Why yes I do. And I will not say, “do not take it”. It is your money, consequently, your choice. I, for example, once bought an expensive jacket with sleeves that were too short, I still wear it, to remind myself that I should spend my money more wisely and buy jackets with longer sleeves. We all spend our money on daft things every so often, so don’t feel bad about that; or do, I don’t mind. Keep to the syllabus. These exams are not like school exams, they want you to pass. After all, you paid for training, if you fail, it makes the trainers look bad, and they don’t want that. Stick to the syllabus like a piece of discarded chewing gum. Parrot like an African Grey. Your job, when sitting the exam, is not to display learning, your job is to regurgitate whatever you were told or read in the syllabus. Regardless of what you think about what you were told, your job is to repeat it back. For multiple choice: don’t think “what is the right answer?” Instead think “what do they think is the right answer?” And watch out for double negatives in questions, these are a useful lazy way of writing questions that trip people up. For written questions you need to remember that the people marking use a set of guidelines. These guidelines tell the marker what type of phrases to expect to see and how many marks to give for each phrase. I harnessed this at university by running through the paper very quickly, regurgitating and parroting words and phrases from the syllabus in the form of a mind- map or outline for each question. If you don’t score it out, it becomes part of your answer. Then I would come back later and finish the answer with written English, knowing that I had probably already met the marking guidelines with the brain dump and that I could not run out of time. You paid. They want to pass you. Make it easy for them to pass you. Write down clearly and concisely whatever it was they told you. Engage your memory, not your brain. Yours Educashunaly, Professor Evil Br ief His t orY OF Time a Fr enchEdition Whatisthemostevilmetricthatisusedin softwaretesting,anddidyouinventit? Q3. FROM Steve Green Dear Steve, Any metric can be used to beat people over the head; as such much malfeasance can be done in their name. “Number of Test Cases” ranks high on the twisted scale because, even when not wielded, simply gazing upon it, casts a spell of malevolence. Consider. One has to posit the existence of a tangible physical entity called a ‘Test Case’ before one can even make sense of the words. This immediately puts the innocent reader in a state where such terrible things can become manifest in this world. Most testers have not read the “Experior Maleficarum”, they have not learned to recognise Continued on page 25
  • 25. 25 Follow us at www.twitter.com/testingclub RegularTesters,getintoSecurityTesting-http:/ /buff.ly/Q9G8Zl Br ief His t orY OF Time a Fr enchEdition Continued from page 24 ‘words of disreputable definition’, and even an Indagator trained in the “Rituale Exploro” can lapse when confronted with primitive heathen words of power such as this. As soon as one posits the existence of a test case it become natural to ask questions such as “how many do we need?” And worse, “how many have you done?” And then you start to think you haven’t done enough, given how many you need to do. And how you do you increase the “Number of Test Cases” done? By making it possible to ‘do’ them faster, by associating each counted ‘Test Case’ with a ‘Test Script’ - this allows you to add cheaper and more ‘Unskilled Testers’ to the project to convert ever more ‘Test Cases’ into the ‘done’ state. And as we all know, once a ‘Test Case’is done. It can never be undone. Otherwise your ‘test coverage’metric and ‘test progress against estimate’ metrics become invalid. Therefore we restrict the validity to the ‘phase’within which their state changes. And so we see that a simple 4 word Whencanapersonsaythattestingisnot requiredforaparticularproduct? Q4. FROM Yogesh Sharma Hi Yogesh, I like questions with flippant answers. So, of course the answer is “Whenever they want”. They can equally say “Bibble Bibble” whenever they want. But I suspect you want a less flippant and more scientific answer (Bwa’haha). My first Google search on “% of IT projects that fail” provided me with a scientific range of “62 - 68”%. I will make this complicated statistic easy for my readers by conclusively stating that 70% of IT projects fail. Therefore, I can conclude that 100% of people on 70% of IT projects can say “Testing is not required for this particular product” and they can hold their heads up high in the hope that the project was doomed anyway. Pretty good odds. Consequently we only have to concern ourselves with the 100% of people involved in 30% of IT projects. We all know that “you can’t test quality into a product” therefore I can use this to conclude that the product will either be quality or it won’t, so we can’t use ‘quality’ as a justification for testing. So basing my judgement on the preceding science; I can say: If a person has the power to cause the project to fail, then they can say “testing is not required”, at the point they make the decision to doom the project. Making rocket science look easy, Oncle E □ invocation leads to the primitive constructs which once were used in the savage times including “Test Phase”, “Test Scripts”, “Unskilled Tester” “Pass/Fail”. Brrr, I get chills just thinking about it. I did not invent this metric. I confess, I have used it, but I didn’t inhale. Frater Evil By Amy Phillips Much of a tester’s work involves telling people that they’ve done something wrong or missed something out. We’re often left chasing incomplete requirements or missing bug fixes. Our approach and outlook can be negative (well we do spend a lot of time expecting things to be broken). Often this is our strength but just sometimes this approach can go against you. I recently held a bug bash to quickly hammer down a load of bugs that weren’t getting prioritised, but that were causing a distraction. We’ve done this before to deal with a steady increase of bugs. This time it was to clear up exceptions in our server logs; mostly problems that didn’t affect the user hence our failure to prioritise fixing, but still a distraction from real problems. Developers love to write code but ask them to fix a bug and you’re on difficult ground. Yes they know it needs doing and yes they know it’s caused by their work but that still doesn’t make it fun. After all when was the last time you looked forward to doing the washing up? With a little bit of creativity you can break Unleash Your Creativity AUTHORPROFILE-AMYPHILLIPS AmyPhillipsisTestLeadatSongkick,a startupprovidingpersonalizedalertsabout livemusicevents.Shehas8yearstestingex- perienceatcompaniesincludingRoyalMail, TheGuardian,andYahoo!Amyispassionate aboutleanprinciplesandstrivestoenable developerstomoveasfastaspossiblewith- outcausingchaos. to focus on new approaches or ideas, but don’t allow them to get burdened down with a negative outlook. Leaving your tester mind-set at the door and turning something that would usually be seen as a chore into a fun, team-led activity can help you succeed. Give it a try next time you face a boring but important meeting, a problem or even just the daily grind. You may not always succeed but you’ll at least have some fun trying! □ the negative cycle and have a fun and productive bug bash (or anything else!). Here’s what we did: • Ask the developers to have some input into which bugs get fixed. Our bug-tracking tool allows people to vote for bugs but any simple voting system would work. Developers rarely get a chance to speak up about which bugs they think should be fixed, so this was a great opportunity for them to fix their pet peeves. • Make it visible. We took over a large whiteboard and generated lots of company-wide support. Use team meetings, newsletters and noticeboards as a way to publicise the event and generate interest. During and after the event put out updates and results. Above all stay focussed on the positives. • Make it fun – We decorated our whiteboard with balloons, this was a bug bash after all! Each bug had a sweet attached to it (to eat during or after depending on how much sugar that fix was going to require!) • Cake – never underestimate the power of cake! Problems and delays create the perfect opportunity NEWSIN BRIEF Sauce Labs is very happy to announce the general avail- ability of OS X, iOS, and Android in the Sauce cloud testing services. Mobile web and desktop web developers can now use our service to run their functional tests on iOS (iPhone, iPad), Android, Safari+OS X, Firefox+OS X, Google Chrome+OS X.  http://bit.ly/ttpapplesauce
  • 26. 26 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag SecondSignofMadness:LookingforTestTools?-http://buff.ly/Q9GdMF Br ief His t orY OF Time a Fr enchEdition NEWSIN BRIEF Perfecto Mobile, A Cloud- Based Mobile App Testing & Monitoring Platform, Raises $15 Mil- lion Series C http://bit.ly/perfectoc By Richard Edgren Good, effective software testing requires methodical hard work in combination with a creative effort. The creativity is needed to get many perspectives (outside explicit requirements), and also to find and judge effective methods for running important tests. There is much written about how to be creative, and a lot of general tips can be useful for testing as well (search for “creativity tricks”, or read any book by Edward de Bono). But even more important is the testers’ environment; creativity comes primarily where it is allowed, and fostered. Swedish philosopher Nils-Eric Sahlin has collected a basic set of environment characteristics1 that make a lot of sense when applied to the software testing profession: generosity, a sense of community, qualifications, cultural diversity, trust and tolerance, equality, curiosity, freedom of spirit, small scale. You are always part of creating this environment; let’s have a look at some of these from a testing perspective. Qualifications Knowledge is needed in order to take fruitful leaps into the unknown. Knowledge of test techniques and heuristics makes it a lot easier to apply effective testing. Knowledge of the technology makes it a lot easier to come up with test ideas. Knowledge of the domain the software will be used in will make it a lot easier to find important problems. Knowledge of the product will make testing faster, and will help you find interacting areas of interest. It is vital to have a generous environment where everybody learns and shares, preferably from different areas. Diversity Different points of view can givwwe radically new ideas when they blend. If all testers found the exact same things, there would be a lot of double work, and no synergy. By using group dynamics, we can take advantage of the strong and weak sides of each team member. For example it might be good to have a member of the test team that always only does what is stated in the online help. Or can a testing team be too diverse? Trust & Tolerance If the testers aren’t trusted, they will probably do just what they are told to. To be creative involves doing new things, which requires trust. Patience to wait for exciting things to happen is needed; trust that a lot of time also needs to be wasted. Creativity under pressure is difficult. Creativity in general is connected to allow ones’ self to make mistakes. This is especially easy in software testing, since mistakes can mimic end users behaviour, and thereby expose issues. “If you’re not prepared to be wrong, you will never come up with anything original.”2 In a test environment with detailed test scripts, it must be OK to side step the instructions, as well as follow them rigidly. Creative testing is about doing REFERENCES 1. http://www.fil.lu.se/sahlin/kreativitet/content. html 2. KenRobinson,http://www.ted.com/index.php/ talks/view/id/66 3. http://en.wikipedia.org/wiki/Flow_(psychology) An Environment for Testing Creativity RikardEdgren,humanistictestersince1998,specializedingeneralitiesliketestingeducationand exploratorytesting.Memberofthink-tankthetesteye.com.Co-authorofSoftwareQualityChar- acteristics,authorofTheLittleBlackBookonTestDesign.TestspecialistatQamcomResearch &Technology,Karlstad,Sweden. AUTHOR PROFILE - RICHARD EDGREN things differently; so different approaches should be embraced. Humour Humour is often about looking at things from a different perspective, to combine things that shouldn’t be. Laughter is fun and generates a good atmosphere where creativity can prosper. That is why you can laugh at developers’ mistakes, but also on missed points in a bug report, or the occasional duplicate bug that you already reported yourself. But most important: If it is fun to work, we do a better job. Discipline It must also be noted that the best ideas often come after a lot of hard work. It might be a bad example, but Edison tried 1000 times before he succeeded with the light bulb. So a creative environment can’t be a place where you just do what you feel like. It takes discipline to learn the product good enough, to learn everything that needs to be learned, and to execute also the boring tests that are needed. Diving into the details and the whole, not far from Csikszentmihalyi’s Flow3 , seems applicable for software testing at its best. Summary The ingredients in this creativity recipe seem simple (I added humour and discipline to Sahlin’s list), but still there aren’t many genuinely creative environments. Sahlin concludes that the unpleasant reason for this is that we, as humans, are driven by the seven deadly sins. I rather believe that many of us somewhere have lost the most important ingredient: Intrinsic motivation.An environment that encourages individuals, that’s where testing creativity can grow. □
  • 27. 27 Follow us at www.twitter.com/testingclub IsthereaRegression‘half-wayhouse’? -http://buff.ly/R0B7gC Br ief His t orY OF Time a Fr enchEdition HOT BLOGS Three Software Testing Books I’d like to read byEvilTester http://bit.ly/3testbooks -- Its official, I have the happiest job in america byMatthewSullivan http://bit.ly/happytesting -- Quality Insurance byMikeMeurs http://bit.ly/Qki0Qs -- The Future Of Mobile Testing bySimonStewart http://bit.ly/futuremobiletesting -- Why I Won’t Go Back byElisabethHendrickson http://bit.ly/iwontgoback -- The Great Agile Testing Quadrants byMaartenvandenEnde http://bit.ly/W5FwWc -- Testers and UX and That’s Not My Job byPeteWalen http://bit.ly/testersandux -- When in a rush, stop and look around byPekkaMarjamäki http://bit.ly/wheninarush -- Can You Really Have Too Many Tests? bySebRose http://bit.ly/toomanyteststtp -- Testing 2.0 byGoogleTestingBlog http://bit.ly/testing2point0 COMMENTCHAT DISCUSS By Karen Johnson Creativity. Mention the word and images of artists painting come to mind. The images conjured up around creativity seem a stark contrast from those of us more computer and business-minded, but the divide between business and creativity doesn’t have to be so wide. When we solve problems, whether those problems are resource constraints or we think of clever ways to test a product, I believe we are being creative: Create Mind Maps A mind map can be a fast and easy way to capture ideas. For me, mind maps seem to work best when my ideas are muddled together, meaning I haven’t quite figured out a structure or a relationship between my ideas. Mind maps seem to work incredibly well when I’m just starting to capture my ideas. As I build a mind map I’m better able to see my ideas take shape; I see subtopics form and cluster; I can see the relationships between ideas. And by identifying those relationships a mind map helps me to organize what I need to do.As an added bonus, I can colour-code ideas to see priority and to direct my focus. For mind maps that I build for my week’s tasks (yes I build mind maps for the week and often use them in client discussions), the color-coding can help me to see when I’m overcommitted. I frequently use color-coding to direct my focus for the day or for my one-on-one meetings. With a mind map, I can see at a glance what I want to accomplish this day or this week in relation to a larger goal. I can add a branch or a node and easily relocate or suppress and expose branches if I want to focus in a certain direction. My mind maps grow with me as my ideas mature, take form, and become clear. And eventually I see the whole presentation or a project or a week (or whatever I’m mind mapping) come together. There is something forgiving about drawing mind maps on my tablet that has helped me to use mind mapping much more than ever before. The ability to pinch and zoom, collapse and expose branches quickly and easily is helpful in a way I was not able to achieve on a desktop computer or with paper. Read One way to generate creative ideas is to read like a fiend and capture your thoughts as you read. Highlight sections in books. Take notes as you move throughout your day. Tag books, magazines and blogs for easy retrieval. Build your own sources of inspiration that you can turn to when you need a spark of inspiration. My office is packed with books. My iPad and Dropbox account have resources I turn to easily even when I’m on the road. If you read about creativity and most notably if you read books written by Alex Osborn (who coined the term brainstorming and authored the book “Applied Imagination”) or read Michael Michalko’s books (“Thinkertoys” and “Creative Thinkering”) or Edward de Bono’s works (author of “Six Thinking Hats”), you realize solving problems is a creative process. You don’t have to draw or paint to be creative. Use a Deck of Cards You never know how you can develop an idea from some seemingly unrelated source to your work. For example, in buying and reviewing two different decks of “creativity cards” one deck called the “Creative Whack Pack” by Roger von Oech and the “Thinkpak” deck by Michael Michalko, I was inspired to collect software testing heuristics (fallible guidelines or rules of thumb for ideas you can then apply to a problem1 ) into a format that can be printed on a deck of index cards2 . Continued on page 28 Re-Creativity:aTestingTipsSpecial Figure1.Beginningamindmap brainstorming mind mapping group mind mapping brainstorming variations 6-3-5 3-12-3 speed thinking pass it on reflective thinking alone time lateral thinking six thinking hats whack pack Phoenix Checklist R: Thinkertoys by Michael Michalko R:http://www.kaner.com/pdfs/ExploringExploratoryTesting.pdf emptying the mind influence alignment map note taking questioning methods & techniques
  • 28. 28 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag ApplyingTheLessonsofRapidTestingIntensive-http://buff.ly/RxgJUD Br ief His t orY OF Time a Fr enchEdition REFERENCES 1. TheEssenceofHeuristics,JamesBach-http:// www.satisfice.com/blog/archives/462 2. Anexampleofwhatsuchcardsmightlooklike canbefoundonAdamBrown’sbloghere:http:// testing.bananalogic.net/playing-cards/ 3. LynnMcKeealistingofheuristicsandmnemon- icshttp://www.qualityperspectives.ca/resourc- es_mnemonics.html 4. MichaelBolton,Article-TestingWithoutaMap -http://www.developsense.com/articles/2005- 01-TestingWithoutAMap.pdf 5. KarenN.Johnson,HeuristicCardDeck.Available asadownloadfrommybloghttp://karennicole- johnson.com/wp-content/uploads/2012/07/ Testing-Mnemonics-as-cards.doc 6. ElisabethHendrickson’scheatsheet-http:// testobsessed.com/2007/02/19/test-heuristics- cheat-sheet/ 7. ContextFreeQuestionsforTesting,Michael Bolton-http://www.developsense.com/ blog/2010/11/context-free-questions-for- testing/) 8. CemKanerandAndyTinkhamonthePhoenix Checklist-www.testingeducation.org/a/explore. pdf 9. ThePhoenixChecklist,RobLambert -http://blogs.imeta.co.uk/RLambert/ar- chive/2008/10/17/is-software-testing-like- being-in-the-cia.aspx Build Checklists Maybe a cheat sheet or a checklist doesn’t sound very creative but it could be. Consider a well-known checklist called The Phoenix Checklist, a list of questions developed by the CIA (Central Intelligence Agency). This is a list I became aware of by reading Michael Michalko’s book, “Thinkertoys.” Why not take a look at the list and see if it helps you. After all, there is value in standing back and seeing if your viewpoint changes, to gain a perspective you didn’t have when you were standing in the same place using the same point of view, trying to solve an issue. How do you use the Phoenix Checklist? Use the checklist like any checklist or heuristic, with judicial prudence. Look at the list. Find a question you can use and see how the questions help you consider a problem from a different viewpoint. I’m not the first tester to discover this checklist, take a look at Michael Bolton’s blog7 and how he used the checklist and made it his own by customizing it. Or how about the work of Cem Kaner and Andy Tinkham who found value in the checklist as well8 . Rob Lambert blogged about the Phoenix Checklist too9 . On the topic of checklists, a book called “The Checklist Manifesto: How to Get Things Right” by Atul Gawande; highlights the value of and use of several checklists. The book is a fast read and helped me to see that a checklist doesn’t have to be rigid, but can be a helpful reminder of items you might overlook when a situation becomes “almost too automatic.” Learn to Brainstorm The word brainstorming is used frequently and sometimes haphazardly. Sometimes it seems as though if two or more people are together and more than one idea surfaces, we say we are “brainstorming.” I’d recommend going back and reading about brainstorming to rediscover the original ideas first published by Alex F. Osborn. Osborn wrote several books and specifically I’ve been reading “Applied Imagination, Principles and Procedures of Creative Problem Solving” and “Unlocking Your Creative Power: How to Use Your Imagination to Brighten Life, to Get Ahead.” Osborn explains why judging an idea too early in the ideation process is a detriment. He illustrates why quantity over quality of ideas is important. Quantity in our critical society seems counterintuitive but Osborn believed that by furthering an idea or by expanding or combining ideas, we could arrive at great ideas. When we shoot down someone’s idea we stifle that person’s flow of ideas and the potential for what may come next. Osborn details how to host a brainstorming session and variations on brainstorming such as having a solo brainstorming session (something I do often as a frequent lone tester and independent consultant.) In delving deeper into heuristics and brainstorming, this path of research brought me back to a book (long referenced within the testing community) called “How to Solve It:ANewAspect of Mathematical Method” by George Polya. While the book on the outside may seem like a guide for math teachers, the list of ways to look at problems from analogies to puzzles has been shaping how I look at issues and most specifically testing challenges in a new light. Play a Game Breaking up problems into smaller problems or trying to bring a spirit of light humour to problems somehow brought me to read the book “Gamestorming: A Playbook for Innovators, Rulebreakers and Changemakers“ by Dave Gray, Sunni Brown and James Macanufo. This book offers a fantastic list of activities that can be used from ice breaking to brainstorming to bringing a group together. Some of the exercises do fine for one person and other activities a more suitable to a group. Summary Having small easy to use tools like card decks and checklists, and being able to use those tools helps me to think of testing ideas, approaches and solutions. With many creative ideas and tools at hand, you can inspire your testers or yourself, or be ready to ask good probing questions in a design session that may prevent a defect from ever being built. If you can do those things, then you’re innovating and you are creative. And you didn’t even have to draw or paint. □ KarenN.Johnsonisasoftwaretestconsultant.Sheisfrequentspeakeratconferences.Karenis acontributingauthortothebook,BeautifulTestingbyO’Reillypublishers.Shehaspublishednu- merousarticlesandblogsaboutherexperienceswithsoftwaretesting.Sheistheco-founderof theWRESTworkshop,moreinformationonWRESTcanbefoundathttp://www.wrestworkshop. com/Home.html.Visitherwebsiteat:http://www.karennjohnson.com AUTHOR PROFILE - KAREN n. johnson Continued from page 27 Using my heuristic card deck helps me. By picking up the deck and peeling through the cards, I always end up with multiple ideas and those ideas unlock my thinking. One idea leads to another, and the next thing I know I’m brainstorming a list of testing ideas. Not all heuristics work in all contexts but I believe most testing heuristics contain a word or a concept that just in mentioning or considering inspires a testing idea. If you read software-testing blogs, you will find many testers have written about heuristics and mnemonics3 4. A mnemonic is a device to help your memory; some testing heuristics have a mnemonic associated with the heuristic. These testers look for commonality or patterns in testing and find a premise they can use repeatedly and by doing so develop a heuristic. Why not tap into those ideas? Or look for patterns or situations in your own testing context and build a heuristic that makes sense to you. You could build your own card deck (mine5 is still a work in progress), or a reference sheet and in creating your own set of heuristics, you will have become more creative and furthered your own skill at solving problems. For example Elisabeth Hendrickson built what she refers to as a “two-page cheat sheet” of testing ideas6 . I find this list helpful. Her cheat sheet is packed with ideas. Figure2.ThinkpakbyMichaelMichalkoand CreativeWhackPackbyRogervonOech Figure3.HeuristicCardDeck
  • 29. 29 Follow us at www.twitter.com/testingclub FollowusonTwitter!@testingclub@testninjas@testingfeeds@stcjobs CREATIVE LEARNING MINDMAP Br ief His t orY OF Time a Fr enchEdition Creative Learning Traditional Learning Certification Degrees Training courses Through people Talk to people! Go to meetups Encourage internal activities Hack days Away days Paired activities / testing Online Communities Through resources Books, publications, blogs... Game Storming Software TestingClub Weekend Testing UTest Challenge Yourself Start a blog Make something Online Coursera Udemy Stanford Online Open University Read plenty Take notes Share what you learn Draw a picture Track Your Learning Journal / blog Write It Down MindMaps How Drawings Tools Calendars Blog Photos Writing Tools Evernote OneNote Set Goals Monthly Read something Research something Write something Article Blog post Book! Tools Approaches Ideas Yearly Attend, organise or do... Training Course Conferences 3-5 Years Vertical Progression Horizontal Chamge Start a business Go independent What's your career path? Meetups Khan Academy Videos Technical Managerial Consultant SME Coach / Trainer Freelance Contract Do something different Read Widely Read deeply Code Academy Podcasts Participate If you are loud, be quiet Take a different route home Talk to someone different Learn something new Write an article Create a video Focus or defocus Ask for help or help someone Go beyond your comfort zone Word Processor Pen and Paper ...many more! Don't rely on your memory alone
  • 30. 30 July 2012 | www.thetestingplanet.com | Use #testingplanet hashtag XSTUDIO XStudio is a free ALM/test management solution allowing to manage requirements/specifications, scrum projects, Automated/manual tests, campaigns and defects. An LGPL SDK is also included to interface with proprietary tests. www.xqual.com ----------------------------------------------- PARASOFT SOATEST Parasoft SOAtest automates web application testing, message/protocol testing, cloud testing and security testing. Parasoft SOAtest and Parasoft Load Test (packaged together) ensure secure, reliable, compliant business processes and seamlessly integrate with Parasoft language products (e.g., Parasoft Jtest) to help teams prevent and detect application-layer defects from the start of the SDLC. Moreover, Parasoft SOAtest integrates with Parasoft Virtualize to provide comprehensive access to traditionally difficult or expensive to access development and test environments. Parasoft SOAtest provides an integrated solution for: • End-to-end testing • Environment management • Quality governance • Process visibility and control. www.parasoft.com ----------------------------------------------- LOADSTORM LoadStorm – The lowest cost and easiest cloud load testing tool. Free account for 25 users. Test up to 100k vusers. Real-time graphs with key performance metrics. www.loadstorm.com ----------------------------------------------- BecomeaTestNinja-www.testninjas.com TEST TOOLS & SOFTWARE Br ief His t orY OF Time a Fr enchEdition THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory GEMINI Gemini brings versatile test management, bug and issue tracking to your team. Sign up to our cloud-based offering or install locally. Join the new generation in software project management with Gemini – no hidden extras or crazy pricing. 3 Users FREE – No Gimmicks – Full Edition. www.geminiplatform.com ----------------------------------------------- TESTRAIL TestRail – Test Case Management Software for QA and Development Teams. Comprehensive web-based test case management software to efficiently manage, track and organize your software testing efforts. http://www.gurock.com/testrail/ ----------------------------------------------- TESTLODGE TestLodge is an online test case management tool that allows you to manage your test plans, requirements, test cases and test runs with ease along with issue tracker integration. www.testlodge.com ----------------------------------------------- TESTOPTIMAL Model-based data-driven test design and test automation to improve test coverage, enable rapid response to changes and reduce test maintenance cost. www.testoptimal.com ----------------------------------------------- ENTERPRISE TESTER Enterprise Tester | Award-winning test management platform offering great features, support, and pricing. Enterprise Tester provides a flexible test management and execution feature set, full coverage from requirements to defects, dashboards and reporting, TQL, and a REST API. It integrates with JIRA, TFS, Enterprise Architect, Selenium and more. Plus, if you are a Not-for- Profit or running an Open Source project you can ask for a FREE community license. Get started for $10 or try Enterprise Tester FREE for 30-days: www.enterprisetester.com/try ----------------------------------------------- KALISTICK Kalistick gives testers a new solution to design efficient test strategies focusing on business risks. Our unique technology analyzes test cases footprints and functional changes to select the most relevant test cases. Discover how to move one step ahead in testing efficiency. www.kalistick.com ----------------------------------------------- TESTWAVE TestWaveisanextgenerationtestmanagementtool implementedasSoftwareasaService(SaaS).Itcan bedeployedinstantlyandyouonlypayforwhatyou use.TestWaveisdesignedforbothTestManagers andTesters,andprovidesrequirements,test planning,testexecutionanddefecttracking.Intuitive graphsreporttestingdatainrealtime.Reduceyour costsandunleashthepowerofSaaSwiththecloud’s firstfullyextensibletestmanagementtool. www.testwave.co.uk ----------------------------------------------- THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory
  • 31. 31 Follow us at www.twitter.com/testingclub CheckouttestingeventsfromMinistryofTesting-http://www.ministryoftesting.com/training-events/ Br ief His t orY OF Time a Fr enchEdition TEST HATS Test Hats are an independent software testing services provider, with offices in the UK and Spain. We provide a full range of testing services including System, Performance and Security testing along with specialised Consultancy and Training. For near-shore testing our Test Lab is fully equipped with a range of desktop and mobile platforms, testing software and tools, allowing us to provide a quality service at a competitive price. Visit our website to learn more about Test Hats and our services. Get in touch today to talk about how we can help test your projects. www.testhats.com ----------------------------------------------- THE TEST PEOPLE The Test People delivers the best, most innovative, highly technical and competitive performance engineering and test service available today. Based upon our extensive experience, TTP can deliver tailored services to address all aspects of the functional and non-functional test lifecycle, including highly specialised performance engineering and test automation services including automated build and continuous integration solutions. TTP are at the forefront of utilising the cloud for test and load environments, with significant experience in open source and the major commercial toolsets whilst also coming armed with our own performance and automation frameworks. www.thetestpeople.com ----------------------------------------------- REVOLUTION IT RevolutionITistheleadingQualityAssuranceand Testing,managementconsultingfirminAsiaPacific. WehelpourclientsdeliverITprojectsandhavecore offeringsacrossProjectManagement,Requirements ManagementandApplicationTesting.Wehaveover 250staffandofficesinMelbourne,Sydney,Brisbane, Canberra,AdelaideandSingapore.Ouroffering includesdeliveryconsulting,methodologies,tool solutionsandtraining.Wehavestrategicpartnerships withHPsoftware,IBMRational,Oracle,Agile AcademyandSAP.WithHPwehavebeentheleading HPSoftwarePlatinumPartnerfor4yearsrunningand theleadingreseller,1stlinetechnicalsupport,training andservicespartner. www.revolutionit.com.au ----------------------------------------------- EUROSTAR EuroSTAR is Europe’s premier software testing conference and has grown to become the largest and most prestigious event on the software testing calendar. Conference attendees can choose from numerous thought-provoking presentations, intensive tutorials, interactive sessions and inspirational keynotes. Plus, visit Europe’s largest software testing exhibition which showcases the leading companies in the industry. We hope you can join us for EuroSTAR 2012 in Amsterdam, The Netherlands from 5 - 8 November for the 20th anniversary of our testing conference! The EuroSTAR Software Testing Conference is where the testing community gathers for an intensive 3-4 days of learning, networking and discussion. www.eurostarconferences.com ----------------------------------------------- COMMUNITIES,CONFERENCES ANDNEWS www.testninjas.com WWW.MINISTRYOFTESTING.COM CO-CREATING SMARTER TESTERS EDUCATION • COLLABORATION EVENTS SOFTWARETESTINGSOLUTIONS THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory THETESTINGPLANETDIRECTORY-GETLISTEDWITHTHESEAWESOMECOMPANIES-thetestingplanet.com/directory
  • 32. OnthenightofOctober23,manyofuslostadearfriendandcolleaguewhenOlaHyltén,fatherof twodaughters,tragicallydiedinacaraccident.Throughouttheworld,Olawasknowntomanyin thetestingcommunityforhiskindness,witandhumour,aswellasforhissupportanddedication tothecontext-driventestingcommunity.Wewhohadtheopportunitytoworkwithhimwill alsorememberhimforhischaracter,strongsenseofethicsandunwaveringdesiretomakethe peoplearoundhimfeelgood.Olahadatruerock´nrollspirit.Heneverjustacceptedasituation becausesomeonesaiditwassoandhewasnotafraidtospeakupwhenhefeltheneededto. Ola’smottowhichhetrulylivedbywas“Ithinkformyself.”Manyofushavealsobeenpartnersin andbenefactorsofhisthinking. Olaworkedhardtoadvancetheunderstandingofexploratorytestingandrelatedpracticesathis workplaceinthelocaltestingcommunityandwastwiceaparticipantattheSwedishWorkshop onExploratoryTesting.Itwasduringoneofthoseworkshopsthattheideaoforganisinga Europeanconferenceoncontext-driventestingwasbornandlittlemorethanayearlater,the inauguralLet’sTestconferencewasheldinStockholm,Sweden,withOlaasconferencechair.As muchasOlalovedtestinghealsolovedmusicandespeciallyRock,sowhynotcombinethetwo ofthemhefigured.FewoftheparticipantsatthatconferenceislikelytoforgetOla’sopening remarkswhichstartedwithAC/DC:s“ForThoseAbouttoRockWeSaluteYou”asOlaentered thestage.Againatruerock´nrollspiritfromamanwithahugeheart.  JohanJonasson& HenrikAndersson Ola Hyltén 1966 - 2012 A Tribute To Ola