Selenium Tips & Tricks, presented at the Tel Aviv Selenium Meetup
12 likes3,898 views
The document provides tips and strategies for using Selenium to write automated tests. It begins with an overview of Selenium and outlines a 10 step process to write successful tests. The steps include defining a test strategy, choosing a programming language, learning Selenium fundamentals, writing the first test, implementing page objects for maintainable code, making tests resilient to failures, adding cross-browser testing, creating an automated feedback loop through continuous integration, and finding additional information resources. The document emphasizes writing reusable, maintainable and resilient tests that can run across browsers to test an application.
Selenium Tips & Tricks, presented at the Tel Aviv Selenium Meetup
- 1. How To Use Selenium, Successfully by Dave Haeffner, @TourDeDave
- 4. Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers. Then package and scale them for you & your team.
- 5. Selenium Overview • What it is — the Reader’s Digest version • What it is and is not good at • IDE vs. Local vs. Remote • Slow, brittle, and hard to maintain?
- 6. Step 1 Define a Test Strategy
- 7. Test Strategy 1. How does your business make money? 2. What features of your application are being used? 3. What browsers are your users using? 4. What things have broken in the app before? Outcome: What to test and which browsers to care about
- 8. Step 2 Pick a Programming Language
- 9. Programming Language • Same language as the app? • Who will own it? • Build a framework or use an existing one? • http://bit.ly/seleniumframeworks
- 11. Selenium Fundamentals • Mimics human action • Uses a few common actions • Works with “locators” Locators tell Selenium which HTML element to interact with
- 12. Common Actions • get(); • findElement(); • click(); //or submit(); • sendKeys(); • isDisplayed();
- 13. Locator Strategies • Class • CSS selectors • ID • Link Text • Partial Link Text • Tag Name • XPath Good locators are: • unique • descriptive • unlikely to change That rules a few of these out
- 14. Locator Strategies • Class • CSS selectors • ID • Link Text • Partial Link Text • Tag Name • XPath Good locators are: • unique • descriptive • unlikely to change That rules a few of these out
- 15. Locator Strategies • Class • CSS selectors • ID • Link Text • Partial Link Text • Tag Name • XPath Good locators are: • unique • descriptive • unlikely to change That rules a few of these out Start with IDs and Classes
- 16. Locator Strategies • Class • CSS selectors • ID • Link Text • Partial Link Text • Tag Name • XPath Good locators are: • unique • descriptive • unlikely to change That rules a few of these out Start with IDs and Classes Use CSS or XPath (with care)
- 17. Locator Strategies • Class • CSS selectors • ID • Link Text • Partial Link Text • Tag Name • XPath CSS vs XPath http://bit.ly/seleniumbenchmarks http://bit.ly/cssxpathexamples
- 18. Finding Quality Locators • Inspect the page • Verify your selection • e.g., FirePath or FireFinder • http://bit.ly/verifyinglocators • Learn through gaming • http://bit.ly/locatorgame • Conversation
- 20. Step 4 Write your first test
- 21. Good Test Anatomy • Write for BDD or xUnit test framework • Test one thing (atomic) • Each test can be run independently (autonomous) • Anyone can understand what it is doing • Group similar tests together
- 22. A Login Example 1. Visit the login form 2. Find the login form’s username field and input text 3. Find the login form’s password field and input text 4. Find the submit button and click it 1. or, find the form and submit it
- 26. Now to find an assertion 1. Login 2. Inspect the page 3. Find a locator 4. Verify it 5. Add it to the test
- 29. Exception Handling • org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"css selector","selector":".flash.error"} • Most common ones you’ll run into: NoSuchElement and StaleElementReferenceError • A list of all WebDriver exceptions: http://bit.ly/se-exceptions-java
- 31. Step 5 Write reusable and maintainable test code
- 32. Page Objects
- 33. Application Under Test Test 1 Test 2 Test 3 Test 4 Test 5Test 1 Test 2 Test 3 Test 4 Test 5 Need to update EVERY test :-(
- 34. Application Under Test Page Object(s) Test 1 Test 2 Test 3 Test 4 Test 5 Need to update JUST the page object :-D
- 35. Let’s look at a page object for login
- 37. And here’s what the test looks like when using it
- 39. Base Page Object
- 40. Selenium Commands Page Object 1 Page Object 2 Page Object 3 Page Object 4 Page Object 5
- 41. Base Page Object Page Object 1 Page Object 2 Page Object 3 Page Object 4 Page Object 5 Selenium Commands • Global reuse • More readable • Insulates you from Selenium API changes http://bit.ly/se-upgrade
- 42. Let’s take a look at a Base Page Object
- 44. And here it is implemented
- 46. How everything fits together Test TestTest Page Object Page Object Base Page Object Tests use page objects Page objects inherits the base page object The base page object wraps your Selenium commands
- 47. Step 6 Make your tests resilient
- 48. Waiting
- 52. Explicit Waits • Specify an amount of time, and an action • Selenium will try repeatedly until either: • The action is completed, or • The amount of time specified has been reached (and throw a timeout exception)
- 54. In the Base page object
- 55. In the DynamicLoading page object
- 57. Step 7 Prep for use
- 58. Test Harness • Simple organizational structure • Central setup and teardown • Configurable at run-time (with sensible defaults) • Reporting & Logging • Parallelization • Test Grouping
- 59. Folder structure
- 60. Central setup/teardown More on JUnit Rules: http://bit.ly/junit-rules
- 61. Simple config with defaults
- 62. Reporting & Logging • Machine readable e.g., JUnit XML • Human readable e.g., screenshots, failure message, stack trace Fantastic Test Report Tool http://bit.ly/se-reporter (Allure Framework)
- 63. Parallelization • In code • Through your test runner • Through your Continuous Integration (CI) server #protip Enforce random order execution of tests http://bit.ly/junit-random-order Recommended approach: http://bit.ly/mvn-surefire
- 64. Test Grouping • Metadata (a.k.a. Categories) • Enables “test packs” • Some category ideas • wip • shallow • deep • story number More info: bit.ly/junit-categories
- 65. Step 8 Add in cross-browser execution
- 67. Chrome
- 68. Grid Grid Hub Browser Tests All done with the Selenium Standalone Server Just requires additional runtime flags Grid Node Grid Node Grid Node Browser Browser
- 69. Grid Hub Node(s)
- 70. Grid More on Selenium Grid http://bit.ly/se-grid-docs http://bit.ly/se-grid-post http://bit.ly/se-grid-extras http://bit.ly/se-grid-scaler
- 71. Sauce Labs Sauce Labs Browser Tests
- 72. Sauce Labs Additional Considerations - Test name - Pass/Fail status - Secure tunnel More on Sauce: http://bit.ly/sauce-platforms http://bit.ly/sauce-post http://bit.ly/sauce-tutorial-java
- 74. Step 9 Build an automated feedback loop
- 75. Feedback loops • The goal: Find failures early and often • Done with continuous integration and notifications • Notifications e.g., remote: Email, chat, SMS in-person: audio/visual, public shaming
- 76. Code Committed Unit/Integ. (pass?) Deploy to autom. test server (success?) Run automated tests (pass?) Deploy to next env. yes yes yes Notify team if no Code Promotion Bonus points: stop the line
- 77. Simple CI configuration 1. Create a Job 2. Pull In Your Test Code 3. Set up Build Triggers 4. Configure Build steps 5. Configure Test Reports 6. Set up Notifications 7. Run Tests & View The Results 8. High-five your neighbor
- 79. Step 10 Find information on your own http://bit.ly/se-info-slides http://bit.ly/se-info-video
- 80. Steps to solve the puzzle 1. Define a Test Strategy 2. Pick a programming language 3. Use Selenium Fundamentals 4. Write Your First Test 5. Write re-usable and maintainable test code 6. Make your tests resilient 7. Package your tests into a framework 8. Add in cross-browser execution 9. Build an automated feedback loop 10. Find information on your own
- 81. Write business valuable tests that are reusable, maintainable and resilient across all relevant browsers. Then package them and scale them for you & your team.
- 82. –Dave Haeffner “You may think your puzzle is unique. But really, everyone is trying to solve the same puzzle. Yours is just configured differently — and it’s solvable”
- 84. Selenium Tips & Tricks by Dave Haeffner @TourDeDave
- 86. etc.
- 87. Headless w/ Xvfb Tip 38
- 91. Headless w/ GhostDriver Tip 38
- 93. Step 2 - Start PhantomJS w/ WebDriver flag Step 1 - Download PhantomJS http://phantomjs.org/download.html Step 3 - Connect Your Test to PhantomJS using Selenium Remote NOTE: You can also connect PhantomJS to a Selenium Grid
- 95. HTTP Status Codes Tip 17
- 96. Configuration • Use a proxy server to capture the traffic from your Selenium test(s) • Find the status code for the action you’re interested in (e.g., visiting a URL) • Assert that the status code is what you expect
- 101. Blacklisting Tip 66
- 102. Configuration • Use a proxy server to manipulate the traffic from your Selenium test(s) • Identify third-party resources that are slow to load (which could negatively impact your tests) and blacklist them (e.g., make it so they don’t load)
- 105. Load Testing Tip 68
- 106. Configuration • Use a proxy server to capture the traffic from your Selenium test(s) • Convert the HTTP Archive into a JMeter JMX file • Run the new JMX file with JMeter to enact load on your application (modify as needed)
- 109. Code adapted from ruby-jmeter example from flood.io
- 112. Broken Image Checking Tip 67
- 113. Option 1 - Proxy Server Option 2 - HTTP Library Option 3 - JavaScript
- 114. Option 1: Proxy Server
- 116. Option 2: HTTP Library
- 118. Option 3: JavaScript
- 121. Configuration • Use Selenium to trigger the forgot password workflow (which sends an e-mail to a Gmail account) and keep the session active • Retrieve the e-mail and it’s contents (e.g., a URL) through the Gmail API • Launch the URL from the e-mail using the Selenium session
- 124. A/B Testing Tip 12
- 127. Example Explanation • There are 3 states available in the example on the- internet. Each state has different header text. • The control has the text ‘A/B Test Control’ • The variation has the text ‘A/B Test Variation 1’ • Outside of the split test, the text says ‘No A/B Test’
- 128. Configuration • You can easily opt-out of A/B tests by forging a cookie or appending a query to the URL • This way you get a known state of the page which isn’t likely to change without your knowledge
- 131. Downloading a File Tips 2, 8, & 15
- 132. Two approaches • Configure Selenium to download to local disk, and delete the file when done • Use an HTTP library, perform a HEAD request, and check the headers for the correct content type & length. A HEAD request with an HTTP library is an order of magnitude faster than downloading with Selenium since you’re just checking the headers. And, it can be used in tandem with Selenium.
- 133. With Selenium
- 135. With an HTTP library
- 136. With an HTTP library (for secure files)
- 147. See the video at http://bit.ly/se-growl-video
- 148. Visual Testing
- 149. Some Write-ups http://bit.ly/se-visual-1 Getting Started http://bit.ly/se-visual-2 False Positives part 1 http://bit.ly/se-visual-3 False Positives part 2 http://bit.ly/se-visual-4 Add Visual Testing To Your Existing Tests http://bit.ly/se-visual-5 Add Visual Testing To Your BDD Tests
- 150. A Robot On Every Desk ™