Moving 65,000 Microsofties to DevOps with Visual Studio Team Services
- 3. I work here
- 6. I work here
- 7. Visual Studio Team Services Continuous Delivery for Every Team, Every App, Every Platform Continuous Delivery Agile Planning Dashboards Kanban Boards Taskboards Build and Test Git Source Control Modern Code Workflow Continuous Integration Continuous Testing Package Management Open Source Compliance Plan + Track Monitor + Learn Release Develop + Test Delivery Deployment of app and infrastructure PaaS, IaaS and Containers Monitoring Telemetry Diagnostics Analysis
- 10. Sprint 1 August 2010 Sprint 29 June 2012 VSTS Preview Sprint 64 April 2014 VSTS GA Sprint 67 June 2014 1ES Sprint 92 Nov 2015 Extensions Sprint 102 June 2016 GVFS Sprint 124 Sep 2016
- 14. Increase flow of value Shorten cycle times Continuously Improve
- 17. • Original estimate • Completed hours • Lines of Code • Team capacity • Team burndown • Team velocity • # of bugs found Things we don’t watch• Acquisition • Engagement • Satisfaction • Churn • Feature Usage Usage • Time to Detect • Time to Communicate • Time to Mitigate • Customer Impact • Incident Prevention Items • Aging Live Site Problems • SLA per Customer • Customer Support Metrics Live Site Health • Time to Build • Time to Self Test • Time to Deploy • Time to Learn Velocity
- 18. Found one of the top customers with low availability. Proactively reached out and resolved their issue.
- 23. • • • • • • •
- 25. Before • Redundant alerts for same the issue • Needed to set right thresholds and tune often • Stateless alerts contributed to further noise After • Every alert must be actionable and represent a real issue with the system. • Alerts should create a sense of urgency – false alerts dilutes that
- 26. Health model in action • 3 errors for memory and performance • All 3 related to same code defect • APM component mapped to feature team • Auto-dialer engaged Global DRI Eliminated alert noise ~928 alerts per week to ~22 and reduced DRI escalations by ~56%
- 27. • •
- 28. “One time” deployment commands in OneNote, email Set-Options “-p 0” Imagine a dozen more steps like that… And then…someone misses a step half way through We once broke pre-production for a day
- 29.
- 30. 1 2 3 4 5
- 33. Double blind test Full disclosure at or near end vs. Share tactics & lessons learned Continued evolution Assume Breach - Use War Games to the learn attacks and practice response
- 35. Alignment Autonomy “Let’s try to give our teams three things…. Autonomy, Mastery, Purpose”
- 36. Phone VS/PS Phone VS/PS Phone VS/PS Phone VS/PS Phone VS/PSUnique PS accounts and workflows 1 year migration plan WDG Product Studio Account WDG VSTS Account
- 37. Can you add all of my custom workflow fields to the bug template that everyone uses? Can you add a new work item type that is only used by my feature team? Is it okay if I track my team’s hardware purchases in VSTS? Can we store every Watson report as a bug even if we’re not sure it needs to be fixed? Is it okay if I create new rules that everyone has to follow for how to track dependencies? Is it cool if I add a few hundred new default values? I prefer Tags, let’s use those! I prefer Keywords, let’s use those! Yes!
- 40. Work Management Information Tools Coding Tools Testing Tools Build Tools Debugging Tools Developer Time Source: Internal Microsoft Telemetry analysis of 4700 engineers in WDG
- 44. Teams Physical team rooms Cross discipline 10-12 people Self managing Clear charter and goals Intact for 12-18 months Own features in production Own deployment of features
- 45. UI API Data
- 46. UI API Data
- 47. Employee choice, not manager driven Typically <20% change, but 100% get to make a choice Cross-pollinate talent and micro-culture Sticky Note Exercise - Self Forming Teams
- 48. Sprint 3 weeks 1 Plan 3 sprints 3 Season 6 months 6 Scenario 18 months Teams are responsible for the detail Leadership is responsible for driving the big picture
- 49. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan CY17 H1
- 50. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 51. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 52. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 53. Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 18 month scenario 6 month plan
- 54. TIME VALUE
- 55. TIME VALUE
- 58. What we accomplished Progressive Deployment Week 1 Week 2 Week 3 Week 1 Week 2 Week 3Week 2 Week 3 Sprint 98Sprint 97 Sprint 99 The sprint plan
- 68. engineers on your team# 5 ?x =
- 69. Rule: If your bug count exceeds your bug cap… stop working on new features until you’re back under the cap. 5 50x =10
- 71. Code Test & Stabilize Code Test & Stabilize Code Complete Planning
- 73. • Tests that anyone can run anywhere (inc production) • Shifted to unit tests from automated functional tests • Core tests run before pull request • Fast and 100% reliable build and test is critical • Rolling tests run after commit
- 75. PR Validations (Impacted Test runs and tests) PR Validations Build and static tools PR Creation Create PR VSO.PR Build+L0 VSO.L1 OnPrem.SelfTest RM.SelfTest Manual Tfs.SelfHost VSO.CredScan Cred Scanner Policheck CI – Required Runs CI - Test (Release Defs) CI - Build (Build Defs) Branch Update Push VSO.CI SPS.SelfTest Tfs.SelfTest Onprem.SelfTest Report RM.SelfTest Report Spsext.SelfTest Tfs.SelfHost Report VSO.Packaging.CI VSO.SDKSample. CI
- 76. Customer Focussed Production First Mindset Team Autonomy + Enterprise Alignment Shift Left Quality Infrastructure as Flexible Resource
- 78. Shared abstraction layer Single master branch, multiple release branches Update 1 Update 2 Update N Visual Studio Team Services Team Foundation Server One code base with multiple delivery streams
- 79. Shared Platform Services (SPS) North Central TFS SU1 North Central AT AT AT JA JA JA Blob TFS SU7 Australia TFS SU0 West Central Containerized Services
- 80. • • • • • • If you are starting out today and cloud native, consider PaaS, Service Fabric + Azure Functions • We need to ship same code to on-prem & cloud
- 82. After • 3-week sprints • Vertical teams • Team rooms • Continual Planning & Learning • PM & Engineering • Continual customer engagement • Everyone in master • 8-12 person teams • Publicly shared roadmap • Zero debt • Mockups in PPT • Inner source • Flattened organization hierarchy • User satisfaction determines success • Features shipped every sprint Before • 4-6 month milestones • Horizontal teams • Personal offices • Long planning cycles • PM, Dev, Test • Yearly customer engagement • Feature branches • 20+ person teams • Secret roadmap • Bug debt • 100 page spec documents • Private repositories • Deep organizational hierarchy • Success is a measure of install numbers • Features shipped once a year