'Where Exploration And Automation Meet: Getting The Most From Automated Functional Tests' by Andy Kemp

Editor's Notes

• #3: 3 minWill be presenting this at Eurostart.Feedack appreciated.Andy KempStarted testing 15 or so years agoJoined TW 6 years ago.Works as a tester and project manager.Now product manager.
• #4: 5 minDefined by the Context driven school.To test well you can’t separate out test design and execution.Exploratory testing is about learning about the application and learning about testing while executing tests.Freedom to discover important information about an app.Essential in today’s world where people expect there applications to be usable and stable.
• #5: 5 minMachines can do repetitive tasks quicker than humans.About checking. Takes its input from testing. You test a hypothesis, once you have proved or disproved it you should automate to check this stays the same.Automation can allow you to quickly check that the basic functionality of you application has not regressed.Essential in today’s world when you need to release continuously.
• #6: 3 minToolsDrivers (selenium,webdriver, sahi, watir)Frameworks/Executable requirements (cucumber, concordian, fitnesseetc)Twist. Which I will use in my demos, as this is the tool I am most comfortable withPatternsBDDATDDPage object
• #7: 2 minCollaborative, iterative developmentValues automated and exploratory testing
• #8: 2 minA Common pattern I see is.Automate then explore“automation frees testers to carry out manual exploratory testing”Often described as orthogonal activities.There is nothing to say that exploratory testing can’t be automated.
• #9: 2 minAutomation is expensiveManual testing is expensive
• #10: 3 minLeverage you automation to save time exploringFeedback into your tests, continuous improvement of test designYou probably already do some of this in your teamsAs a community we should be sharing patterns and practices in this area
• #11: 2 minEfficiency of exploratory testingRepeatability of bugsFeedback loop into automationBetter test, better test code
• #12: 10 minIts important to structure your tests well.Building blocks to quickly generate new test scenarios.Demo in Twist
• #13: 5 to 10 minuteMix up manual and automated steps in a single test.Why waste time manually running steps you have already automated?Set up your app to where you want to manually exploreDemo in Twist
• #14: 5 to 10 minMake you tests data driven where it makes sense.It should be easy to switch data and rerun the test over and over as you think over new ways to stress the systemDemo in Twist
• #15: 3 minTest maintenance is importantYour test suite must be modular and well maintained for this to work.Set up and tear down to return your system to a happy state.AlanTitchmarsh talks about the coffee cup gardener. Every day spend half an hour in the garden with a cup of coffee helps keep the garden maintained.You can do this with you test suites, get a coffee, poke around, tidy up the odd test.
• #16: 3 minYou need to easily access the right test at the right time.Organise with tags or folders so you know where to look for different types of tests.Find tests and use them as spring boards for new ideas.Check results won’t trigger the same kind of thinking
• #17: 5 minsWell written tests, DSL etcIf your tests represent your domain, using them as building blocks will be easier
• #18: 2 minSum upThis is just to get you thinking and asking the questionsWhat can you leverage on your project to support your exploratory testing?Some examplesPete Houghton – Guardian blog about automating tests on their APIs in an exploratory test.Lisa Crispin sometimes Tweets about thisAny more I unearth before the conferenceWhat do you do?