Developing a test automation strategy by Brian Bayer
4 likes1,307 views
The document discusses developing a test automation strategy and considerations for regression testing, automation, and frameworks. It addresses why regression testing is important, factors for the automation strategy like business needs and ROI, and frameworks to consider like Cucumber, Robot Framework, and Spock. The strategy determines test maintenance and fits within the organization, development process, and technology stacks.
![*** Testcases ***
Login Should Succeed When the Correct
Username and Password are Entered
Enter Username AUser
Enter Password TestPass
Click the Login Button
Login is Successful
*** Keywords ***
Setup Test
Open Browser http://zdiles.chaosnet.org/ googlechrome
Maximize Browser Window
Enter Username [Arguments] ${username}
Input Text uname ${username}
Login Is Successful
Page Should Contain Welcome](https://image.slidesharecdn.com/developingatestautomationstrategybrianbayer-160221220125/85/Developing-a-test-automation-strategy-by-Brian-Bayer-53-320.jpg)
Developing a test automation strategy by Brian Bayer
- 2. why are we here?
- 3. regression testing the greatest cost of development
- 4. regression testing you have already paid for this at least once
- 6. why regression test? quality ensures what was working, still works
- 7. why regression test? quality risk money lost revenue, market, contractual ensures what was working, still works deliverables, release, schedule
- 8. why regression test? a project without thorough and continuous regression testing is like a road trip where you do not look out the window until the end of the trip
- 9. why automate? (vs. manual)
- 10. why automate? (vs. manual) TimeMoney
- 11. why automate? (vs. manual) Money
- 12. why automate? (vs. manual) Time
- 13. automation strategy what to consider... tests are living specifications of the system
- 14. automation strategy what to consider... ARE THE TESTS GOING TO BE USED?
- 15. automation strategy what to consider... business needs return on investment organizational structure and culture
- 16. automation strategy business needs... determine areas of focus of the automation
- 17. automation strategy return on investment... limits and prioritizes the areas of focus of the automation determines at which level to automate a test determines the automation framework
- 18. determines the success or failure determines the speed, ability & openness to change determines the communication determines the ability to learn automation strategy organization and culture...
- 19. automation strategy organization and culture... determines who uses and maintains the tests determines the automation framework too
- 20. automation strategy what else to consider...
- 21. automation strategy what else to consider... development process and lifecycle maintenance of the tests technology stacks
- 22. automation strategy development process and lifecycle... legacy (system without tests) to greenfields
- 23. automation strategy development process and lifecycle... system under test development process methodology: waterfall agile xp ad hoc is automation development part of the same process?
- 24. automation strategy development process and lifecycle... is the automator part of the development team? how much access to the system developers?
- 25. automation strategy maintenance of the tests... who and how much can they maintain? how are the system developers involved?
- 26. automation strategy maintenance of the tests... how timely can/must tests be maintained what reporting needed to maintain tests
- 27. automation strategy finally, technology stacks... languages and technology already used unit and integration testing source code control, continuous integration devops or operations
- 30. who is going to use the tests?
- 32. where to automate? https://msdn.microsoft.com “Its essential point is that you should have many more low-level unit tests than high level end-to-end tests running through a GUI.” Martin Fowler
- 33. where to automate? UNIT LEVEL INTEGRATION LEVEL GUI INTEGRATION LEVEL XPtest driven development ACCEPTANCE LEVEL BDD
- 34. ACCEPTANCE LEVEL BDD where to automate? acceptance tests only if the business is engaged and going to use them ACCEPTANCE LEVEL BDD
- 35. where to automate? what to consider... DRY
- 36. look for low-hanging fruit risk-based testing what computers do well, humans not so much fragility of test where to automate? what to consider...
- 37. which automation framework to use?
- 38. which automation framework to use? Gherkin/Cucumber Robot Framework Selenium Webdriver Spock/Geb JUnit Jasmine TestNG
- 39. which automation framework to use? what to consider... ORGANIZATIONAL STRUCTURE AND CULTURE
- 40. which automation framework to use? what to consider... who is reading the tests? who is writing and maintaining the tests? balance of right tool with existing technology, knowledge, and ability to learn ORGANIZATIONAL STRUCTURE AND CULTURE
- 41. which automation framework to use? what else to consider... natural language for high level specification goal is to develop tests NOT framework and...
- 42. grids...
- 43. integration...
- 45. reporting...
- 46. teh sexy
- 48. but JRuby (JVM), Java, Groovy, JavaScript, .NET (using SpecFlow) libraries (Selenium, Watir) Ruby has the richest ecosystem with gems gherkin/cucumber generally ruby based...
- 49. Feature: Cash Withdrawal Scenario: Successful withdrawal from an account in credit Given my account has been credited with $100 When I withdraw $20 Then $20 should be dispensed And the balance of my account should be $80 Given /^my account has been credited with (#{CAPTURE_CASH_AMOUNT})$/ do |amount| my_account.credit(amount) end
- 51. also runs on Jython (JVM) & IronPython (.NET) libraries (Selenium, Watir, SOAP, etc.) keyword based supports “given when then” supports tabular/data driven robot framework python based...
- 52. *** Settings *** Library Selenium Library Suite Setup Start Selenium Server Suite Teardown Stop Selenium Server Test Setup Setup Test Test Teardown Close All Browsers
- 53. *** Testcases *** Login Should Succeed When the Correct Username and Password are Entered Enter Username AUser Enter Password TestPass Click the Login Button Login is Successful *** Keywords *** Setup Test Open Browser http://zdiles.chaosnet.org/ googlechrome Maximize Browser Window Enter Username [Arguments] ${username} Input Text uname ${username} Login Is Successful Page Should Contain Welcome
- 54. *** Testcases *** Login Should Succeed When the Correct Username and Password are Entered Given I Enter Username AUser And I Enter Password TestPass When I Click the Login Button Then Login is Successful given when then
- 55. spock/geb/groovy
- 57. developing in java or grails?
- 58. groovy-based browser automation tool built on Selenium WebDriver works with JUnit, TestNG, Spock page models and jQuery selector syntax spock/geb/groovy geb...
- 59. groovy-based test and specification framework supports “given when then” tabular/data driven spock/geb/groovy spock...
- 60. import geb.spock.GebSpec class GebHomepageSpec extends GebSpec { def "can access The Book of Geb via homepage"() { given:”One is on the Geb Home Page” to GebHomePage when: “I click the jQuery Link” highlights.jQueryLikeApi.click() then: “I should be at the jQuery Section” highlights.jQueryLikeApi.isSelected } }
- 61. import geb.Page class GebHomePage extends Page { static url = "http://gebish.org" static at = { title == "Geb - Very Groovy Browser Automation" } static content = { highlights { $("#sidebar .sidemenu").module(HighlightsModule) } sectionTitles { $("#main h1")*.text() } } }
- 62. FRAMEWORKS
- 63. other options...
- 64. raw Selenium (Java) excel spreadsheet data driven scripts script capture record/playback flat files in source control other options...
- 65. which automation framework to use? the simplest thing that could possibly work
- 66. tests are specifications of the system...
- 67. how does it fit?
- 68. how does it fit... design your approach into the organization
- 69. how does it fit... find the gaps and tight squeezes
- 70. the specifics...
- 71. the mechanics...
- 72. so what’s left?
- 73. documentation of the strategy...
- 74. execution of the strategy...
- 76. estimation...
- 77. resources...