The Bumpy Road Towards Continuous Delivery
0 likes509 views
This document summarizes Benjamin Bischoff's presentation on the challenges trivago faced in establishing continuous delivery of their hotel search product releases. Initially, trivago relied heavily on manual testing which led to long release cycles and unstable features. They created a centralized automation team to develop test frameworks, pipelines and ownership models. This empowered teams to integrate automated testing earlier. While technical hurdles around infrastructure and frameworks remained, changing mindsets around the value of automation through education and communication was also critical to their progress towards continuous delivery.
The Bumpy Road Towards Continuous Delivery
- 1. The Bumpy Road Towards Continuous Delivery Benjamin Bischoff Web Test Automation Engineer SeleniumConf Berlin 2017
- 2. Topics 1. About trivago 2. Main Product 3. Releases 4. Central Automation Team 5. Challenges 6. Summary 2
- 4. trivago • Founded 2005 • First hotel meta search in Germany • Headquartered in Düsseldorf 4
- 5. trivago’s Mission Statement “To be the traveller's first and independent source of information for finding the ideal hotel at the lowest rate”
- 7. Main Product
- 9. 9 We are forwarding you to the booking site deal. Please double-check the hotel information before making your hotel booking. OTA Business Model
- 10. 55 Markets 10
- 11. So many things…
- 12. We Talk About… ...Manual and Automated Testing ...Deployment and Test Pipelines ...Frameworks and Technologies ...Documentation and Knowledge Sharing 12 Why?
- 13. User Value
- 14. Hotel Search Releases The Past
- 15. 15 The Past Feature Development Manual Tests Merge Automated Tests / Release
- 16. 16 Problems • Repetitive manual work • Old features could break • Possible long release cycles • Frequent rollbacks • Automated tests did not have any real effect Feature Development Manual tests Merge Automated Tests / Release
- 17. Hotel Search Releases The Goal
- 18. 18 The Past Feature Development Manual Tests Merge Automated Tests / Release
- 19. 19 The Goal Feature Development Manual Tests Automated Tests Merge / Release
- 20. 20 Advantages • Stable new features • Quicker release cycles • Easier refactoring • Critical paths are covered • New features cannot break older ones Feature Development Manual Tests Automated Tests Merge / Release
- 21. The Team
- 22. Release & Test Automation Team • Release Engineering & Test Automation • Maintain the test framework and pipelines • Support all teams using our platforms • Decrease release time • Remove repetitive work • Improve stability 22
- 23. Test Automation Team Mission Statement “Make automated testing a widely used and reliable technique at trivago.”
- 24. Old Automation Approach • Old framework was not flexible enough • No clear guidelines • No full test suite • No clear test result visibility • No test ownership 24
- 25. RTA TA Decisions Who writes and owns the tests? How are the tests structured? Where should the tests be located? How / when should the tests be run? 25 Empower QA Clear test guidelines Inside the SUT Test framework and pipelines
- 27. 27 Test Framework • User friendly • Stable and consistent • Clearly documented • Easy to run tests • Enable parallel execution • Support application specific features
- 28. 28 Test Infrastructure • In-house Selenium grid • Lots of maintenance • Expensive • External Selenium grid In-house grid External grid Hard to set up (static) Instantly available Lots of maintenance Less maintenance Hard to scale Scaling built in Limited OS / Browser combinations Huge number of combinations possible Less expensive? More expensive?
- 30. 30 “Automated tests will replace all manual testing” ”They want to turn QA into developers” “My test is done, I have nothing else to do.” ”I have no time for automation” Bias
- 31. 31 Bias • Extensive documentation • Workshops • “Automation Time” • Make sure everyone is on the same page! • Testing is a team effort! “Automated tests are there to replace all manual testing” ”They want to turn QA into developers” “My test is done, I have nothing else to do.” ”I have no time for automation”
- 32. 32 Increase Trust In Tests • Dashboards and reporting • Extensive logging • Early flakiness detection • Test qualification job • Slack notifications • Duty managers
- 33. Project Status
- 34. 34 What We Achieved • Test pipelines exist • Test suite of 200 scenarios • Automatic tests on PRs and branches • Kibana dashboards • Full test reports • Clear ownership
- 35. 35 What We Are Working On • Block PRs • Even clearer reporting • Further education • Test framework architecture and speed • Working together with feature developers
- 36. Summary
- 37. 37 Core Values
- 38. Key Points Trust Unwavering Focus Power of Proof Fanatic Learning 38 Trustful Automation Clear Ownership Test Result Visibility Communication Why?
- 39. User Value
- 40. @BischoffDev Web Test Automation Engineer Benjamin Bischoff Thank you!
Editor's Notes
- #10: redirect the user to the local or international OTA or Hotel cost-per-click
- #13: Agile methodologies…
- #16: Feature implementation Pull request Manual testing Merging of PR Manual and automated integration testing Test failures are added as Jira comment
- #19: Feature implementation Pull request Manual testing Merging of PR Manual and automated integration testing Test failures are added as Jira comment
- #20: Feature implementation QA tests the feature Pull request Automated tests are triggered Tests block PR on failure Merging of PR Release
- #31: “Soft skills are the hardest you can learn” Ashley Hunsberger Meaghan Lewis
- #39: Team effort!