Speedup ios release
Download as PPTX, PDF1 like120 views
The document discusses the challenges of increasing release speed for iOS apps from once per month to 3 times per day at a large company. It outlines how the mobile team has grown from 15 to 150 engineers. Key challenges include improving test reliability, speed, and stability to support much more frequent releases. Strategies discussed include increasing test coverage through layout tests and scenario tests, optimizing compiler and test framework speed, running tests in parallel through distributed building and testing, and ensuring stability of the testing infrastructure, tools, environments and distributed testing process. The speaker's company created an open source project called Bluepill, inspired by the Matrix, to help address these challenges through parallel testing on multiple simulators and automatic test grouping.
Speedup ios release
- 1. Keqiu Hu Speed Up iOS Releases
- 2. 2 whoami Software Engineer @ LinkedIn Flagship infrastructure team
- 4. 4 Background Old days One centralized mobile team ~15 iOS engineers 1 mobile release per month
- 5. Past: One centralized mobile team ~15 iOS engineers < 1 mobile release per month 5 Background
- 6. 6 Background Now Decentralized mobile team 150 iOS engineers 2000 commits/month 3x3
- 7. 7 3x3? Release 3 times a day Commit to publish time < 3 hours
- 8. 8 3x3? Release 3 times a day Commit to publish time < 3 hours
- 10. Reliability
- 11. 11 Test Coverage
- 12. Layout Tests Stress test views with different strings 12 Test Coverage
- 13. Scenario Tests Replace manual testing 13 Test Coverage
- 14. Scenario Tests Replace manual testing 14 Test Coverage
- 15. Push beta build to company employees for internal testing 15 Dogfood
- 16. Speed
- 17. 17 Test Automation 7 days to 19 hours (5 hour compilation + 14 hours testing)
- 18. Speed up Swift compiling (19 hours -> 15 hours) 18 Compiler
- 19. Speed up Swift compiling 19 Compiler
- 20. Speed up iOS testing framework (15 hours -> 4 hours) 20 Test Framework
- 21. Distributed building and testing (4 hours to 1 hour) 21 Distributed build!
- 22. Stability
- 23. Precommit -> Code base -> Post commit 23 Stability Pre-commit Code base Post- commit
- 24. 24 Stability Testing infrastructure stability Tooling stability
- 25. Hermetic testing environment 25 Environment Stability
- 26. 26 Test Stability 99.9%1000 = ? 99% 95% 90% 80% 50%
- 27. 27 Test Stability 99.9%1000 = 36.7% 99% 95% 90% 80% 50%
- 28. 28 Trunk Guardian Flaky tests are worse than no tests Disable all flaky tests
- 29. Synchronization between steps 29 Test Framework Stability
- 30. Synchronization between steps 30 Test Framework Stability
- 31. 31 Distributed Testing Stability Running tests on 10 machines Each node has a reliability of 95% and 95%^10 = 60%
- 32. 32 Distributed Testing Stability Running tests on 10 machines Each node has a reliability of 95% and 95%^10 = 60% Capacity requirement
- 33. 33 Distributed Testing Stability Running tests on 10 machines Each node has a reliability of 95% and 95%^10 = 60% Capacity requirement
- 34. Based on facebook’s xctool Python wrapper 34 Project Hydra
- 35. Facebook’s xctool was declared EOL No access to simulator APIs 35 Project Hydra
- 36. The project name is inspired by the Matrix’s Bluepill 36 Project Bluepill
- 37. Objective-c code base on top of CoreSimulator and other Apple frameworks Running tests in paralle via multiple simulators Automatically packing tests into groups with similar running time Retrying when simulator hangs or crashes. 37 Project Bluepill
- 39. Write a config: { "app": "./Sample.app", # Relative path or abs path "scheme-path": "./SampleAppTestScheme.xcscheme", # Relative path or abs path "output-dir": "./build/" # Relative path or abs path } Run! ./bluepill –c config.json 39 Project Bluepill
- 40. Question?
Editor's Notes
- #2: Revolutionized How many of you release your app once per month/week/day?
- #4: ----- Meeting Notes (3/1/17 09:56) ----- background -> scale of the team -> continuous delivery in the past and now
- #6: However, we release IFF all regression tests pass. But in reality that is not the case. Same as in school xxx. Errors in homework As we approach the release day (cut-off date). It is exponential. Bad quality -> open RC PM have to carefully plan their features (miss one -> screw up) brainstorm.
- #7: Mobilize Vertical teams Consistent UI and same vision for the all platforms
- #8: Production quality app to internal -> Code compilation, testing, ipa validation -> 3 hours Get rid of manual testing
- #11: Make sure the app produced by the continuous delivery has a high quality -> higher crash-free rate, high production quality
- #12: Past -> unit test Now -> test pyramid
- #13: All views are rendered based on the models, layout test framework helps exhaust all input strings to make sure xxx
- #14: Man -> manual, machine -> automation Why chose KIF -> no others and kif is fast and no ruby Talk more about this in stability section
- #15: KIF is not enough Slow / Verbose and most importantly stable
- #16: Monitor crash rate in background Without provisioning files With enough test coverage and dogfooding, we achieved a super reliable continuous delivery pipeline.
- #17: How many of you are using Swift in your company?! Intrepid bold But we were brave Longer, crashes, no incremental build Latest Swift and Xcode is troublesome and most big companies refuse to onboard Swift as their primary language INCLUDING APPLE
- #19: By end of this exercise, we ended up with 1h release build time and 20 minutes local dev time
- #20: 20 minutes is not fast enough for local dev! What can we do next?? Hardware!
- #22: The savior is distributed build & test