Self-Generating Test Artifacts for Selenium/WebDriver
Download as PPTX, PDF6 likes4,351 views
The document discusses issues with maintaining automated tests when a website undergoes frequent changes. It proposes generating page object models on every build to account for changes as they happen. This avoids failures from id, type, or navigation path changes. Developers must fix type changes before check-in. With regeneration, tests absorb insignificant changes and the test pass rate improves from identifying issues within 24 hours without noise. The approach provides a model to generate additional tests around standards, errors, forms and security. While reported defects did not decrease, the solution allowed increased velocity and faster feature addition.
Self-Generating Test Artifacts for Selenium/WebDriver
- 1. Page Object Generation Lose the maintenance, increase the productivity
- 2. The Problem Failuresguide our work The product is ever-changing Developers do not communicate changes proactively (or at all) 40% of our time is spent accounting for these changes Who can analyze 300 test failures every night?
- 3. What Changes? Htmlid (i.e. “Locator”) Element type Major navigation path Each will break most automated tests
- 4. Html id Ifthe id changes (e.g. to signinField), the test will break The element will no longer exist
- 5. Element Type Thelink is changed from a Link to Button WebDriver will still look for a link
- 6. Navigation Path Pages are added Popups are introduced Buttons and forms are split up across pages UI Look and feel “reset” WebDriver scripts don’t stand a chance— you just start over
- 8. Wish List for a Solution Build in a mechanism for change communication Account for the changes AS THEY HAPPEN, not reactively Tighter integration with development Pass rate needs to be 97% or better, with all failures accounted for within 24 hours
- 9. Page Object Generation Generate the pages on every build Web Controls are mapped to specific WebElement types If the type of an object changes in a way that breaks automation, the whole build fails
- 10. Let’s See It!
- 11. What you get A page, containing every element in the UI Each element is aware of its Tab Including every Rich WebElement A Fields object, usable by those writing Test Scripts
- 12. The New Change Cycle
- 13. The New Change Cycle •Significant underlying type (Text Input to Radio Button) •Insignificant underlying type (Button to Link) Developer Change •id changes for localization •Compiler flags type change—breakage •Compiler ignores type change (Both are IClickable) Regenerate Model •Page auto-updates id change—no breakage •Tester doesn’t know—fixed before dev check-in •Tester doesn’t know—Framework “absorbs” the change Test •Tester doesn’t know—id is s String, stored in one place
- 14. When change happens Html id changes Who cares? Generation process normalizes names Unit test enforces uniqueness Element type changes Developer fixes prior to check-in Navigation path changes More rare, but failures show up within 24 hours Without all the noise, issues are easier to spot, analyze, and fix
- 15. Where do you go now? You have a model of your system Use it to “analyze itself” The all-table test Role-based security tests Algorithms to Iterate through every page looking for standards compliance 500 errors Forms Security issues (code injection, XSS etc)
- 16. Bottom Line We found 7 major regressions in 2008, the last year of the “old” platform 12 in 2009 with >500 test cases 20 in 2010 with >1100 test cases More than 50 in 2011 with 1700 test cases Customer reported defects did NOT go down, because… Velocity increased so much, features were added at a much faster clip
- 17. Questions?