Distributed agile testing_for_enterprises

Editor's Notes

• #3: Anand:I am a Tester. My interest and passion for software quality and test automation has taken me on many interesting journeys in my career of over 11 years in Software testing, and overall > 14 years in the IT field. In this time period, I have had opportunities to work for organizations like WebMD, Borland, Microsoft and AmberPoint. Along with core testing activities, I have done product support, client interactions, established processes, system administration … basically whatever it needs to get the work done. In ThoughtWorkssinceover 2 years, my profile is of a QA. I am an hands-on software tester, and am involved in all aspects of (manual, automation) testing, training, consulting and practices.
• #4: AnandThis session is about teams, which, due to various reasons that we will touch upon later, are not seated under the same roof, in the same room. These teams are almost self-sufficient in a way … each team has developers that are building functionality, and QA team members that are testing what is being built. In addition, these teams have business analysts to guide them and maybe project managers to make sure development is on track and issues are identified and mitigated at the right time, with the correct people.That said, because the teams are split in different locations, be it same building / city, or different cities / countries / continents, everyone is still working on the same product. Thus, integration of the functionality developed by each team into one product is crucial to be done correctly. That means there is more to testing than what is done by each team, for what they develop.
• #5: Anand
• #6: AnandThis session is about teams, which, due to various reasons that we will touch upon later, are not seated under the same roof, in the same room. These teams are almost self-sufficient in a way … each team has developers that are building functionality, and QA team members that are testing what is being built. In addition, these teams have business analysts to guide them and maybe project managers to make sure development is on track and issues are identified and mitigated at the right time, with the correct people.That said, because the teams are split in different locations, be it same building / city, or different cities / countries / continents, everyone is still working on the same product. Thus, integration of the functionality developed by each team into one product is crucial to be done correctly. That means there is more to testing than what is done by each team, for what they develop.
• #7: AnandThe tips and strategies we are going to talk about are based on:our experiences, what we have seen, done well, learnt from what has not gone wellNO “ONE-SIZE FITS ALL” SOLUTION!!Practices should be applied within the context.EVOLUTION IS THE KEY! - cannot build a bridge by laying its surface at the same time as building its foundation - keep the big picture in mind
• #8: Why?? - Business objectiveAnandRemain the same …..Rapid releases – to cash-in on Time to marketGood quality productWhat does it mean for IT organizations?
• #9: Manish
• #10: What??? Agile in Software DeliveryCan be done to accomplish the “Why”?Manish
• #11: How? To answer the Why and the What!ManishIn our earlier webinar On "Emerging Paradigms of Testing" KristanVingrys spoke about an inside out approach to testing.Lets recap the principles that matter for implementing "The Inside Out approach".We are going to extend these principle in the context of distributed team in this webinar and talk about practices,tools and tips that allow us to use the principle on distributed teamsThe first principle is "Building the quality In".We believe testing should focus on building quality in rather than "testing it in".Remove the cost of fixing the defects don’t just reduce the cost.Find mistakes early and prevent them from becoming prevalent.Tests become executable specification and drive the development.The next principle is "Involving Everyone".Get better tests through diverse input and the team looking after the tests.Also stops over the wall syndrome that results into gap in expectation between different roles and the resulting surprises.Another Principle is about generating instant feedback. This helps in providing predictability in the software development cycle by finding and fixing issues closer to the event that caused the problem.ATDD,Running Test in a CI build pipeline,pairing on the dev box are some of the practices that help achieve this.Tests are an asset of the product allowing reuse across various projectsFast Delivery into Production - Tests should not become last hurdle to cross by testing throughout the projectClear and Consistent view of testing across projectsSimple reporting to help make decisions and navigate the projectBusiness optimize value -Not just Insurance, FIT for purpose,help drive new features and functionalityBefore start looking into how these principles apply,lets look at the reason why distributed team exist?
• #12: ManishDistributed teams are fact of life.Some of the reasons why distributed team existGlobalization - Organizations trying to expand into emerging markets.Also there is a need for localization for the marketThere is an increasing pressure to increase the value delivered for every dollor spentFor some teams 24*7 testing is essential to help rapid releases and provide rapid feedback from test.Rapid Globalization is happening as a result of acquisitions and Mergers for companies to remain competitiveTeam size - Larger projects mean larger team.It may not be possible to locate everyone in same physical space
• #13: ManishShared understanding of projects,goals,vision,status across teams in different locationTimely Decisions - time zone differences mean that you will have to wait unless the stakeholder is present.Defecttriaging can't be done instantaneously.Trust and rapport because of physical separationVisibility in the progress of what each team is doingCommon and consistent way of working.for example Part of the team doing ATDD and rest traditional wayWhile these challenges exist for colocated team only these become much harder for distributed teams because of Lack of communication bandwidthIncreased noise because of physical separationCultural issues and time zone difference
• #14: AnandLets talk about some of the practices you could potentially adopt on your teams to make Distributed Testing more effective.Mindset, Toolset, ATDD, Test Automation, Environments (NFRs), Retrospectives, Defect
• #15: AnandIndividual’s humanpsycology plays a huge part when we interact with others. We most likely would react differently with people in tough situations when they are f2f Vs when in a different location.Here are a few things that can help bridge this gap:Follow KISS (Keep It Simple Silly!) principleKeep an open mindCross-pollination by frequent rotationBe positiveTrust your team – be an optimist than a pessimist ONE PRODUCT, ONE TEAM!
• #16: AnandPeople over process!Not talking f2f with anyone is a pain. Especially if communication is only over emails and status updates and reports. You don’t understand the thought process the other is going through and vice versa.So … what can you do to make it easier:Talk …. Don’t write -> example: GSI case: sending emails to ppl in the same roomPick up the phoneSchedule regular catch-up callsVideo conferencingAlternate daily / weekly schedule to accommodate late / early time callsThis makes it fair for all teamsIncreases trustEveryone feels valuableVideo transcripts – status, demo – with transcripts / logs / annotations to enable searchingClear way to express test for Team understanding – Create Test DSLs, pictorial when ambiguousPairing, info sharing, trainings, webex, vnc, skype, IM (share OVE example)
• #25: AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
• #26: AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
• #27: AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
• #28: AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
• #29: Anand1. Test code should be production quality2. Automation configurable to run against different environments3. Use design patterns4. Proper abstraction layers (tools, browser agnostic, libraries)5. Extensible6. CI7. screenshots on failures8. Logging9. Video recording9. Proper logging – test * product9. Incremental refactoring, 9. Dev pairing – quality of test code, testability, system perspective, 9. Evolve framework, 9. Decouple test data from test implementation, 9. No Copy Paste, 9. Tools & Utilities
• #30: ManishToolSetsAny toolset in a distributed team should aid toCrisp & Clear Communication∫No Extra overhead of traceability∫Common tools for consistent usage across location∫Avoid information islands∫Flexibleand should work with each other
• #31: ManishDSL - shared understanding of test across role.helps creating executable specification.Toolscucumber,FIT,TWISTTagging - Allows tests to align to features,allows to run test only for the features that are changing, categorize them from speed of execution and in general allows test to be categorized and run in multiple conceivable waysVideo Recording and test snaphots for test failures allows clear communicationSupport for CI - support for ant,maven helps test to be hooked up in CI pipelineMultiple drivers - Tools should allow support for multiple drivers to run test through various interfaces - Web applications, webservices without changing the way you express tests - TWIST,In other cases custom abstraction layers need to be created.Exploratory testing support - session recording with annotated videos,screenshots,semi automated execution to improve efficiency - TWIST has an excellent support for thisCustomizations
• #32: ManishCI tool helps team across locations a common way to build test and deploy applicationA common source code repository for both source code and test code and a common CI buildpipeline is essential for team working across distributed location.
• #33: ManishProject Collaboration Tool> Requirement + Project + Test> Common dashboard> Accessible to everyoneKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
• #34: ManishDeterministic and Repeatable self contained stubbed environment are essential-Many times third party application provide interesting challenges for testing,specially when environments are centrally managed Often not available 24*7 ,some time they are under development as well, Responses from them can’t be controlled to execute specific test execution path.Third party applications are stubbed out.When communications are down helps team to continue with testing. Redundancy - Multiple teams means need for multiple showcases,demo,testing sessions these typically lock the environment and teams have to wait.Also some times you want to run your test experiment in isolation from others.Coordinating to schedule environments is difficult for teams in different time zone and locationSame images for building environment (ex: virtualization) no environment difference means overhead of understanding environmental difference is reducedSeparate automation execution environment - not affected by what others are testing or changing an environment or its configuratioTest automation sandpit to help authoring of test scriptsDedicated performance test environment
• #35: Manish: Primary focus: test what is developed in the same location.know what testing activity needs to happen in which location?what is to be tested and from where can that be tested most effectively?ex: functional testing, regression testing, story testing, exploratory testing = done best from where the development is happeningLoad / perf testing = best from where the full production-live environment will be available.team member rotationon-site representationunderstand cultural impactskillset of people on the grounddeployment process
• #36: ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
• #37: ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
• #38: ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
• #39: ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
• #40: AnandIPM:With all team membersPlan purpose and priorityTeams bondPer iteration – NOT big bangIdentify Features / functionality cutting across teamsShowcases:Product Understand more perceptions / other perspectivesFeatures / functionality cutting across teamsProduct meeting stakeholder’s need and expectationsRetro:Retrospectives for distributed teams play a crucial role. They are the time specifically put aside to reflect on how the team is performing and what can be done to improve. Video conferencingEveryone should be there> Check safetyBe open, not judgementalNo finger pointingEveryone is of the understanding that each team member gave 100% Create action items and assign ownersAct on action items!Followup on action items in the next retro> Tools like – ideaboardz,
• #41: AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
• #42: AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
• #43: AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
• #44: AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
• #45: AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
• #46: AnandMindset, Toolset, ATDD, Test Automation, Environments (NFRs), Retrospectives, DefectMindsetEnvironmentsCommunicationDistributing WorkToolsetTest AutomationATDDReporting & MetricsDefect reportingNFRsRetrospectives
• #47: QuestionsHow do you manage test suites in distributed teams? - categorize tests Run in build pipeline Reduce noise from failing tests (quarantined suite, dependent tests) Consolidate overlapping tests- Not Cast in stone- Deprecate Non-applicable tests2. you mention cross-pollination and rotations.  when are the best times in the project for testers to travel?3. do you have any suggestions for creating a common global testing environment to prevent confusion caused by executing tests in non-equivalent configurations?4. you mention that when you distribute the test work that you should think through what kind of testing you do in each location.  what are the important factors to consider?5. Do you have any tips for NFR testing in distributed teams?6. What is bad about Testing Center of Excellence / offshore testing?