Agile Testing Agile Ottawa April 2015

Editor's Notes

• #3: Software Value is contextual, it varies by project, organization and team business value of the software is most important thing So we test Why because Tested code is better code better code is more reliable more reliable code makes customers happier happier customers make sure the team still gets paid Can I ship it? When will we be done? What risks do we see?
• #4: Based on the basics of testing, it applies the mindset Of course this is simple to say but hard to do
• #5: You can’t test quality in, you have to build it in - throwing code over the wall doesn’t work Dev, test, product management, ops, and support all test in different ways We test assumptions as well This does not mean people are interchangable Continuous testing ensures continuous progress
• #6: Per build, per story, per iteration, per release Learning skills agile practices what to do better next time Instead of the test function acting as gate keepers or the quality police - we ask how can we help? Focus on delivering value to the customer - if they aren’t using it it isn’t valuable
• #7: Challenge requirements to get to the true needs
• #8: Before a separate test phase If it is a regression, with automated testing - weekly, daily, hourly, per commit If it is complex - found by testers inside the sprint it the code is being built Not a month later when devs have switched tasks and will have to spelunk the code to figure out what they were doing To generate a safety net that enable developers to change the code with confidence To create real business impacts
• #10: Not this Things don’t work as well if everyone has separate priorities Handoffs are in place team to team which leads to ...
• #13: Everyone tests - not just people with test or quality in their title Functional lines should blur Developers are ‘Test Infected’ All roles actively collaborate The whole team is responsible for delivering a product with quality built in
• #14: As with many things agile - it is about the philosophy and mindset Applied in the context of your projects There are no magic bullets Except at the Texas School Book Depository
• #15: Feedback The fundamental job of a tester is to provide valuable information Value Results oriented, customer focused Comm F2F, Skype, Phone, IM Don’t be document driven Keep a customer focus - how can I help? all users of the feedback Dev Proj. mgmt Ops Support Documentation etc
• #16: Use a investigative journalist metaphor Report observations, and their impact Add context Examine multiple points of view
• #17: Learn intent of features Learn who uses the features Need to know the organizations priorities Devs are awesome at building things, they need focus - test gets to take the holistic view and provide valuable contrast in their thinking Systems Thinking De Bono’s Six Thinking Hats Beginner’s Mind
• #18: Customer Has real needs, make their life better Dev Coding isn’t easy With empathy for the project We have timelines and costs to maintain
• #19: Remember they are invested in their work as well Provide and gather context Why is this important? What are your assumptions? What are the other persons assumptions?
• #20: Best Work They care about that Some people are not effective communicators Benefit of the Doubt If the feedback is starting to cause conflict Are they ill? Overtired?
• #21: Questioning what problem is being solved? how will I know it works? who uses this? what’s the worst thing that could happen? Reflection So that means? Clarification What do you mean by .. Summarisation To rephrase .. I understood .. Feedback wrap Why am I focussing so much on feedback? because
• #22: You need to maintain a good relationship even if your particular objective gets blocked You need to work with these people tomorrow as well
• #23: What does good design look like? Are there code maintainability challenges? Database design and DB query skills Business analysis skills The skills required will depend on your context You can they talk about data persistence and disaster recovery scenarios You can talk about the DB triggers that create behaviour in the system You can understand the challenges of cache invalidation There is value in being able to read code, and SQL, to find the intent of a developer’s change
• #24: The time required to run manual tests will only increase as more are written This has produced a great deal of debate on the interwebs Ultimately I think more knowledge is a good thing For a tester it allows you to find new classes of risk to your product
• #25: Also called Generalizing Specialists So lets look at building a team with cross functional skills
• #26: UX testing Test feedback Test data management Test design Etc
• #27: System architecture Technical writing BDD Story mapping Impact mapping Etc
• #28: So you want a team who can can code with quality, can test effectively, and can automate tests Think of it as a matrix of competencies So some testers can program and some devs can test - the important thing is that the team has the skills to deliver
• #29: Here we move on to test automation
• #30: To deal with change effectively you need a test automation plan Note, you can start without automation, but you should plan your strategy for building it out It is expensive, but should have ROI
• #31: You still need manual testers, because you really want to free them from regression testing to be more valuable to your team
• #32: We’re talking about automation but you’ll still need manual testers To illustrate I’ll add the concept of testing vs. checking Sapient testing
• #33: We should look at what can be automated - James Bach and Michael Bolton propose Testing vs Checking as a useful model You still need manual testers to actually test, vs performing checks that can be automated Sapient testing
• #34: Another way of looking at it - automation can tackle the repetitive know knows I know when I log in with a valid password I should get to the index page For the rest of it humans are required In dev, test, and the rest of the project The example James and Michael use You don’t call compiling a program ‘automated programming’ programming is something humans do Similarly, they call what computers can do ‘automated checking’ Testing is something humans do
• #35: Back to automation - while I can’t cover specifics here I can present a model As you go up the pyramid, tests are more expensive Time to run More maintenance You want a larger amount of tests at the lower levels - but it is important to test many different ways Having a broader approach is often more effective than focusing on one method as thoroughly as possible
• #36: Contrast this to a common anti pattern The SW test ice cream cone Here we optimize for manual testing
• #38: Here is the now classic agile testing quadrants These show options inside a quadrant You will need to tailor testing to your team and product However, I will point out that it is useful to touch a system multiple ways vs putting all your concentration on 1 or 2 types of testing you can then catch multiple classes of bugs the 80/20 rule applies
• #39: This is Gojko Adzic’s take on the testing quadrants. I like this one as it takes into account the concept of Checking and Testing.
• #41: Similarly to automation I can’t go into depth here You need both It complements automated testing to find different classes of bugs
• #43: That is Ad hoc testing - ‘let’s give it a go’
• #45: You can use your past experience E.G. this is on google slides I still expect it to do similar things to power point Next is a heuristic you know intuitively
• #48: Tired Ted, Impatient Ian .. Landmark tour Look at Session Based Test Management from James Bach Not a product recommendation
• #50: Borrowing a phrase from Paul Gerrard It may not be called testing Get testers involved earlier in the software development process What if we had a conversation before things were even coded?
• #51: F16 fighting falcon story from SBE mach 3 why
• #53: Gojko Adzic’s software quality model We can test and gather or give feedback at all these levels - either manually or via automation Looking at the pyramid we immediately realize that as a software delivery team we can only assure the 3 bottom levels of the pyramid and to assure our product is Useful and Successful we need feedback from the final customer
• #54: How do we shift left?
• #55: Defines scope from business goals Are we done yet? Gathers key examples What does this feature look like? Refines examples Concrete examples of system behaviour Automates testing of the examples Leading to ‘Living Documentation’
• #56: like Test Driven Development (TDD) Captures examples of desired system behaviour via conversations Automates testing of those examples Uses a Given, When, Then format As they both involve executable specifications, these two practices are very complementary
• #57: It isn’t about the tools It’s about the conversations To explore examples and behaviour To gain a shared understanding Focus on delivering value instead of documents or automation
• #58: The Goal is To Effectively Implement features,
• #59: Stories are required for the workshop basis of the discussion Defines the intent of the feature the workshop elaborates the story
• #61: You find the examples discussed in the workshop - examples of concrete behaviour
• #63: On to the examples This Defines a story’s scope Makes it testable Give you the key details Business logic Boundary conditions Important - Don’t create an example for everything You’ll get lost in the details E.G. Great idea to kill Amazon - increase sales by offering free book shipping book plus washing machine? - no book to Papua New Guinea? - no Book to US - yes Book to Canada - yes Here we have specific testable behaviour
• #64: Doesn’t this look a lot like a test? If you have a set of these examples it defines the feature Most importantly Biz, Dev, Test had a conversation - they have a shared understanding of the feature and its goals
• #65: A shared understanding saves a lot of documentation - lets take a vacation photo as an example A photo You may say ‘That’s cute’ I remember a river behind the house in Nova Scotia where I had dinner with family friends Our host runs a Christmas tree farm, my daughter found the water and we had to play at the bank for at least a half hour You wouldn’t know that because you were not there Similarly in a workshop you will have picked up all sorts of details that are not documented - but you will understand the feature And testers will know from the beginning, instead of during the transition from dev to test
• #66: The conversation is the most important thing. By collaborating instead of handing off documents - AKA shelfware - we have increased contextual knowledge to deliver the right thing
• #68: - Requirements are not frozen - change is accepted - Priorities change - This is usual - Try to make change as efficient as possible