Strategies in continuous delivery
Download as PPTX, PDF3 likes1,989 views
The document discusses Wix's strategies for continuous delivery. It notes that Wix does over 1500 deployments, 470 A/B tests, and 200 feature toggles from January to June 2013. Wix practices continuous delivery through test-driven development, small iterative releases, feature toggles, A/B testing, and automated deployments. This allows developers to deploy new features quickly while minimizing risk.
Strategies in continuous delivery
- 1. Strategies In Continuous Delivery Learn the “magic” behind the scenes Aviran Mordo Server Group Manager @ Wix @aviranm http://www.linkedin.com/in/aviran 21:34
- 3. Wix in Numbers • Over 35,000,000 users – Adding over 1,000,000 new users each month • Static storage is over 150TB of data – Adding over 1TB of files every day • 3 Data centers + 2 Clouds (Google AE, Amazon) – Around 300 servers • Over 40,000,000 Server API calls per day • ~400 people work at wix – Over 100 people in R&D 21:34
- 4. Is Doing Multiple Deployments A Day Takes Guts ? 21:34
- 5. 21:34 From Jan’ 2013 – Jun’ 2013 • 1500 • 470 • 200
- 6. 21:34 From Jan’ 2013 – Jun’2013 • 1500 Deployments • 470 A/B Tests • 200 Feature Toggles
- 7. Continuous Delivery at Wix • Abandon “VERSION” paradigm – move feature centric life • Make small and frequent release as soon as possible • Automate everything – TDD/CI/CD • Measure Everything –A/B test every new feature –Monitor real KPIs (business, not CPU) 21:34
- 8. Continuous Delivery at Wix • Empower the developer –The developer is responsible from product idea to 1M active users –Remove every obstacle in the developer’s path –Big cultural change from waterfall – affects the whole company 21:34
- 9. Test Driven Development • No new code is pushed to Git without being fully tested • Before fixing a bug first write a test to reproduce the bug • Cover legacy (untested) systems with Integration tests 21:34
- 10. Test Driven Development • What people think is the impact on development –TDD slows down development –With TDD we write more code (product + test code). • Current Test Count (U-Tests + IT-Tests) – over 10,000 21:34
- 11. Test Driven Development • Actual impact on development –We develop products faster –Removes fear of change –Easier to enter some one else’s project –Do we really need QA? (Yes, they code tests) –Writing a feature is 10-30% slower, 45-90% less bugs –50% faster to reach production. –Considerably faster time to fix bugs 21:34
- 12. Guidelines for successful TDD • Tests should run on project checkout to a random computer. • Tests that cannot be debugged on a developer machine will never consistently run for any period of time • Tests should run fast • Tests have to be readable – They are the project’s specs 21:34
- 14. Feature Toggles • Everyone develops on the Trunk • Every piece of code can get to production at anytime 21:34
- 15. Feature Toggle to the rescue • Unused new code can go to production – no harm done • Operational new code goes with a guard – use new or old code by feature toggle 21:34
- 16. Usage example Simple “if” statement in your code 21:34
- 17. Feature Toggle Strategies (gradual expose users) • Company employees • Specific users or group of users • Percentage of traffic • By GEO • By Language • By user-agent • User Profile based • By context (site id or some kind of hash on site id) 21:34
- 18. Feature Toggle Override • By specific server – Used to test system load – New database flows/migration – Refactoring that may affect performance and memory usage • By Url parameter – Enable internal testing – Product acceptance – Faking GEO • By FT cookie value – Testing – When working with API on a single page application 21:34
- 20. A/B Tests 21:34
- 21. A/B Test • Every new feature is A/B tested • We open the new feature to a % of users – Define KPIs to check if the new feature is better or worse – If it is better, we keep it – If worse, we check why and improve – If we find flaws, the impact is just for % of our users (kind of Feature Toggle) 21:34
- 22. An interesting site effect on product • How many times did you have the conversion “what is better”? – Put the menu on top / on the side • Well, how about building both and A/B Testing? 21:34
- 23. Marking users with toss value in a cookie • Anonymous user – Toss is randomly determined – Can not guarantee persistent experience if changing browser • Registered User – Toss is determined by the user ID – Guarantee toss persistency across browsers – Allows setting additional tossing criteria (for example new users only) – Only use this for sections that a user has to be authenticated 21:34
- 24. • Do not mix anonymous and registered tests • AB test parentage of users with optional filters – New Users Only (Registered users only) – By language – By GEO – By Browser – user-agent – OS – Any other criteria you have on your users 21:34
- 25. A/B Test Features • A/B Test Override – Allows setting a value of a test for validation – Helps support experience what users experiencing • Override methods – Via URL parameter – Via cookie • Start/Stop Test • Pause tests • Bots always get “A”21:34
- 28. Refactoring • Before refactoring make sure everything is covered with tests – Legacy code usually covered by IT tests • Refactor from inside out – Small iterations with tests – Refactor small methods - make sure the tests don’t break – Deploy often • Re-write from the outside in – Write from scratch – Code duplication sometimes needed (temporary) – Protected by Feature Toggle21:34
- 29. Code branch 21:34 New Code Old Code FT Opene d Yes No
- 30. DB Schema Changes Without Downtime • Adding columns – Use another table link by primary key – Use blob field for schema flexibility • Removing fields – Stop using. Do not do any DB schema changes 21:34
- 31. New DB schema • Plan a lazy migration path controlled by feature toggle 1. Write to old / Read from old 2. Write to both / Read from old 3. Write to both / Read from new, fallback to old 4. Write to new / Read from new, fallback to old 5. Eagerly migrate data in the background 6. Write to new / Read from new 21:34
- 32. Gradual Deployment 21:34 • Assume two components • We shutdown one and install on it the new version. It is not active yet • Do self test • Activate the new server it is passes self test • Continue deploying the other servers, a few at a time, checking each one with self test A 1.1 B 1.1 A 1.1 B 1.2 A 1.1A 1.1 B 1.1B 1.1 A 1.1A 1.1 B 1.1 B 1.2 A 1.1 B 1.2 A 1.1A 1.1 B 1.1 B 1.2 A 1.1 B 1.1A 1.1A 1.1 B 1.1B 1.2
- 33. Self Test / Post Deployment Test After each server deployment run a self test before deploying the next server. • Checking server configuration and topology – Make sure database is accessible (DB connection string) – Is the schema the one I expect – Access required local resources (data files, other config files, templates, etc’) – Access remote resources – RPC / REST endpoints reachable and operational • Server will refuse requests unless it passes the self test • Allow a way to skip self test (and continue deployment)21:34
- 35. Backward and Forward compatible • Assume two components • We release a new version of one • Now Rollback the other… 21:34 A 1.1 B 1.2 A 1.2 B 1.1A 1.1A 1.1 B 1.1 B 1.2 A 1.2A 1.1 B 1.1B 1.1 A 1.1 B 1.1A 1.1A 1.1 B 1.1B 1.1 A 1.0 A 1.2A 1.1 B 1.2B 1.1 B 1.2 A 1.2 A 1.2A 1.1 B 1.2B 1.1 B 1.0
- 36. Production Visibility 21:34 Prepare for a rainy day
- 37. How does it work – CD Practices • Test driven development • Small Development Iterations • Backwards and Forwards compatible • Gradual Deployment & Self-Test • Feature Toggle • A/B Testing • Exception Classification • Production visibility 21:34
- 38. Where are we today? • We have re-written our flash editor product as an HTML 5 editor – In just 4 months • Introduced Wix 3rd party applications (developers API) – In just 6 weeks • We are easily replacing significant parts of our infrastructure • And we are doing ~10 releases a day! 21:34 0 5 10 15 20 25 30 35 1/1/2013 1/4/2013 1/8/2013 1/11/2013 1/15/2013 1/18/2013 1/22/2013 1/27/2013 1/30/2013 2/3/2013 2/6/2013 2/10/2013 2/13/2013 2/18/2013 2/21/2013 2/25/2013 2/28/2013 3/4/2013 3/7/2013 3/12/2013 3/15/2013 3/19/2013 3/24/2013 4/3/2013 4/7/2013 4/10/2013 4/15/2013 4/19/2013 4/23/2013 4/28/2013 5/1/2013 5/6/2013 5/9/2013 5/13/2013 5/19/2013 5/22/2013 5/26/2013 5/29/2013 6/1/2013 6/4/2013 6/7/2013 6/11/2013 6/16/2013 (blank) Total Total
- 39. 21:34
- 40. 21:34 Aviran Mordo @aviranm http://www.linkedin.com/in/aviran Read more: The Road To Continuous Delivery: http://goo.gl/K6zEK http://www.slideshare.net/aviranwix/strategies-in-continuous-delivery
Editor's Notes
- #2: Today I’m going to tell you some of the strategies we use that allow us to deploy 10 times a day
- #8: Management role is to help developer do its work
- #9: Management role is to help developer do its work
- #14: One of the key components to successful CD
- #19: Full load on a single serverOverride size limitation by setting a cookie on the client
- #22: Link to purchase on the editor was causing drop in conversion because users went there too soon without intent
- #23: Link to purchase on the editor was causing drop in conversion because users went there too soon without intent