Webdriver.io
1 like61 views
This document provides tips for managing the annoyances that can occur with end-to-end browser testing. It discusses employing software developers instead of automation testers to better debug failures, locking down the version of Chrome used for tests, accessing the Selenium container directly if issues occur, and creating forgiving continuous integration pipelines for tests. The goal is to gain the benefits of end-to-end testing while reducing flakiness and debugging difficulties.
![TIP: Employ software developers,
not automation testers
• So you can define semantic selectors:
• Testing your own code:
• button#reset-password
• Testing someone else's code:
• //div[@class="col-md-12"]/
form[@id="login"]/button[2]](https://image.slidesharecdn.com/webdriverio-190404230458/85/Webdriver-io-8-320.jpg)
Webdriver.io
- 1. End-to-end browser testing with a manageable number of annoyances — Steven Noble, 2019
- 2. About me • Creating software since Dick Smith added a Basic cartridge and cassette data storage to the Wizard • Working in the digital economy since 1994: journalism, PR, marketing, consulting, research, etc • React/Node software developer with GorillaStack: we’re like IFTTT for your enterprise cloud infrastructure
- 3. End-to-end browser testing: the theory • Confirm that your services and units work together as required • Test your app the way your users test it — through the user interface • Spend less time on manual testing
- 4. End-to-end browser testing: the reality …that’s all true, but… • It’s really slow • It’s hard to debug • It can be flakey …so here’s how to get those benefits with a manageable number of annoyances
- 5. But first, your stack Headless web browser Chrome only Chrome Firefox etc Browser automation driver Chromedriver Chromedriver Marionette (Firefox) etc Browser automation server DevTools protocol Selenium Browser testing framework Google Puppeteer Webdriver.io Nightwatch TestCafe etc
- 6. It looks like this browser.url('/login'); $('#login-form').waitForVisible(); $('#username').setValue(WDIO_USERNAME); $('#password').setValue(WDIO_PASSWORD); $('#login').click(); $('#dashboard').waitForVisible(); expect(browser.getUrl()).to.equal('/dashboard'); expect($('h1')).getText().to.equal('Dashboard'); expect($('#header').getText()) .to.include(`Welcome ${WDIO_USERNAME}`);
- 7. TIP: Employ software developers, not automation testers So you can quickly tell whether a failure is a bug in the app or a bug in the test So developers will write or maintain tests whenever they write or maintain features aka, be GorillaStack, not the ABC
- 8. TIP: Employ software developers, not automation testers • So you can define semantic selectors: • Testing your own code: • button#reset-password • Testing someone else's code: • //div[@class="col-md-12"]/ form[@id="login"]/button[2]
- 9. TIP: Employ software developers, not automation testers So you can change the behaviour of the app being tested: • Disable Google Analytics, Rollbar, Intercom etc • Use test keys for Google Recaptcha, Stripe, etc • Execute next steps immediately instead of sending them to a message queue • Load/unload database fixtures
- 10. TIP: Lock down Google Chrome If you are just using the Google Chrome binary that you use for everyday browsing, then you are liable to find: • Test behaviour changes randomly after automatic Chrome version updates • Test behaviour differs on different machines that are running different versions of Chrome Instead, use a Docker container or similar that lets you lock your tests to a specific version of Chrome
- 11. TIP: If you Dockerise Selenium, don’t treat the container as a black box • If you get weird Selenium errors in your tests, start by dropping down a level: • Visit http://localhost:4444/wd/hub/static/resource/ hub.html • Send commands to Selenium directly via its HTTP API • You’ll probably see errors related to a known issues with the official SeleniumHQ containers, with possible solutions in GitHub issues
- 12. TIP: If you Dockerise Selenium, don’t treat the container as a black box • If not, drop down a second level: • docker exec -it <constainerID> /bin/bash • Send commands directly to Selenium via its CLI • You’ll probably see errors related to a known issues with the official SeleniumHQ containers, with possible solutions in GitHub issues
- 13. TIP: Create forgiving pipelines in your CI For example, in Buildkite, we: • Run the WDIO step after all other steps, as they all provide useful feedback more quickly • Make the WDIO step mandatory when merging a feature branch in develop • Make the WDIO step optional in all other scenarios • Make good use of the retry attributes in pipeline.yml