Creating Agile Test Strategies for Larger Enterprises

Editor's Notes

• #4: A few years ago - 2011-2013 A lot of talks were on the topic Testing is Dead. Like I believe James Withacker stated in his EuroSTAR keynote that all new innovations game from Dev and test did not contribute much Many organizations reduced the Test departments, fired the test managers. Recently one of clients ( a dutch bank) told that testers could chose to be BA of Dev. And on the testhuddle a discussion thread is lively today…
• #5: Business agilty Livespan of organisation is reduced – American stock market Start-ups loose mometnum and fail to addapt their services So busines agiltiy is key… What makes you flexible…?
• #8: I was quite trilled by the talk of Janet Gregory on Agile testing in the enterprise. I have been publishing and talking about the same topic. Good to hear someone likes to emphasis this story also. Janet stated that agile wants small co-located teams, but in large organizations this is not always possible. Organization culture is defined by Values, norms, etc.… But also structure, history etc… A blame culture might lead to reluctancy to trying new stuff. When testers are blamed for errors, it might be difficult to make quick changes. In large projects, many projects are concurrently active and not seldom people are managed as resources (iso humans). Typical testing issues in larger enterprises are: Throwing over the wall, not knowing who to ask, integration, testers that are working in more than one team, etc. Very often organization fail to look at release total. Tips Janet gave were: Make a mind map with items to test (to validate the release). Janet likes to use a test matrix to share results Have a focus on dependencies…and discuss them. Janet believes testing is part of the team, but in large organizations sometimes testing is too large to fit in the team. Maybe there is room for a product team, a separate “test” team. e.g with some post-development testing. But beware they should be testing in the next sprint, not the whole release.  Testing is a team problem, but in large organization there are a lot people outside the team (Note the keynote by Diane Larsen- the Boundary Interaction map)
• #10: https://www.techwell.com/techwell-insights/2016/12/reviving-master-test-plan-age-agile
• #11: The final keynote was by Gojko Adzic. Gojko made some bold predictions to shake us awake. In short he claimed that in the near future most software is run in 3rd party software- e.g. Amazon servers etc. As example he told that e.g. Amazon is selling login and user settings services as a service. For a company this means that using this saves you 3-months of development time, and further you save maintenance costs since the functions are maintained by de OPS provider. He states that the OPS in DEVOPS is outsourced this way and the line to production and test will blur. Line between production and test environments will blurs when multi versioning becomes mainstream. Since services are charged per request multiple instances will become quite cheap. Therefore, testing explodes. We need to test multiple instances. Another prediction he made was that apps will be distributed. We are running services rather than apps. This makes the architecture complex, but flexible. We can have quick bug fixing by replacing small Parts of the system. Most software will glue together 3rd party services. The risks are in integration, not in the units. The risk will be that a party is changing a service without telling you. The testing pyramid will inverse. Unit testing will become less important than integration testing or PID. Testing after deployment will be critical and Approval testing becomes more relevant than unit testing.   In such an environment we will work with SLA’s and Gojko addresses that failure budgets become a norm. Maybe it does not matter to have bugs in the system, he suggests, as long as we comply to the SLA. 

• #13: So what is needed? 3 cases of projects that I did in the recent past !
• #17: Scenario advice to not accept: nobody wants this shouldn’t be a surprise
• #18: The way to avoid the second scenario is by putting into place witness process Set of goals that need to be reached by 4 activities. Extensive reviewing Witnessing is an ongoing phase and is all about transparency and working together. Avoid surprises, mitigate risk, clear understanding on quality0 Audit: De leverancier toont aan dat: checklist van uit te voeren activiteiten, -> paper trail klopt! Bewijslast ligt bij leverancier. Demo: periodieke inzage in kwaliteit van product. Toont test werkzaamheden van afgelopen periode, testscenario’s worden afgespeeld en inzicht wordt gegeven van de belangrijkste bevindingen (wat werkt nog niet). -> Demo rapport! Meetings: Strategisch testoverleg, Finding Meeting, Demo -> Kwaliteitsoverzicht Review: Testplannen, testcases, Testresultaten, Non functional testen.
• #22: We do not want a screen, we want a feature
• #23: In large organisations there are many people outside the team
• #27: How to ensure Integration We do: CI/CD MBT UT Automated System test Automated e2e test Interface testing Manual Regression testing Integration sprints Other General trend: Increasing the system (e.g from Units tot Systems) results in less frequent integration, because it becomes harder to test the integration. This has impact on the time-to-market and this insight might lead to targeted improvements
• #28: @Split: Technical tests could be: Exploraotry tests, Bughunting, unit test, ST testing, performance testing, interface testing, Business rule testing Business tests could be AO , e2e-chain, Testing in the large Note the Pi-shaped tester: More than one specialism, which one you chose
• #37: Teams collaborate to deliver each sprint a working product Integration is continue Tests are automated Tests are run from the build server Deployment is hands-off process TDD ensures no backlog in TA Acceptance criteria are clear Feedback loop to create better tests in place Product goes live regularly
• #38: Teams collaborate to deliver each sprint a working product Integration is continue Tests are automated Tests are run from the build server Deployment is hands-off process TDD ensures no backlog in TA Acceptance criteria are clear Feedback loop to create better tests in place Product goes live regularly
• #42: The team was best in traditional testdesign, but this had limited value