Developing an Immune System — The Hard and Soft Skills required to avoid Outages
1 like871 views
The document discusses essential hard and soft skills for avoiding outages in software development, emphasizing the importance of techniques like test-driven development, code analysis, and post-mortem analyses. It highlights the need for automated quality checks, collaboration, and a strong focus on culture to improve processes and outcomes. Ultimately, it advocates for proportional investment in maintenance and improvement to foster creativity and efficiency.
Developing an Immune System — The Hard and Soft Skills required to avoid Outages
- 1. The Hard and Soft Skills Required to Avoid Outages @pascallouis
- 6. Code Bless Ship Maintain Profit! git rm
- 7. Tactics • Entity bound ids (e.g. Id<T>) • Textual ids MWDN-YP89-OLVL-USER • Testable configurations • etc. Code Bless Ship Maintain Profit! git rm
- 8. TDD • Not controversial (anymore) • Living code documentation • Enables collaboration • Technique to encode invariants Code Bless Ship Maintain Profit! git rm
- 9. Gold Tests • Tests which can be changed by a (small) subset of engineering • Enforced via policy or technology Code Bless Ship Maintain Profit! git rm
- 10. Expressive Tests • “Change your language and you change your thoughts” — Karl Albrecht • Can be implementation agnostic Code Bless Ship Maintain Profit! git rm
- 11. ... Given feed PaymentEventFeedListener receives: """ { "payment_id": "EPT-300", "isTivoReplay": false, "merchant": { "token": "m-1" }, ... } """ Then expect table balance_changing_events order by id: | event_type | status | process_attempts | | HOLD | UNPROCESSED | 1 | | CAPTURE | UNPROCESSED | 0 | When then the time is 2012-01-06 17:10:00 And balance changing event queue processes items Then expect table balance_changing_events order by id: | event_type | status | process_attempts | | HOLD | UNPROCESSED | 2 | | CAPTURE | PROCESSED | 1 | Code Bless Ship Maintain Profit! git rm
- 12. Automated Manual Quality or or Time Code Bless Ship Maintain Profit! git rm
- 13. Code Analysis • In theory: static vs dynamic • In practice: pre vs post-production Code Bless Ship Maintain Profit! git rm
- 14. Pre Analysis • Type Checking • Testing, CI • Linters • Forbidden Call Analysis Code Bless Ship Maintain Profit! git rm
- 15. Post Analysis • Logging • Metrics • Invariant Checking Code Bless Ship Maintain Profit! git rm
- 16. Alerting & Reporting Sign Res pon al Precise Imprecise se Immediate Alert Oups! Deferred Report Report Code Bless Ship Maintain Profit! git rm
- 17. Metrics vs Checks OK ? WARNING 1 0 200ms 0ms Code Bless Ship Maintain Profit! git rm
- 18. Code Bless Ship Maintain Profit! git rm
- 19. Post-Mortem • When Anytime there are issues! • Why Learn and avoid mistakes of the past • How Blameless Code Bless Ship Maintain Profit! git rm
- 20. Post-Mortem • Go through the timeline • The Good, The Bad and the Ugly • Action Items Code Bless Ship Maintain Profit! git rm
- 21. Root Cause Analysis Code Bless Ship Maintain Profit! git rm
- 22. Code Bless Ship Maintain Profit! git rm
- 23. Proportional Investing • When you lose N hours to maintenance, you spend an equivalent N hours on improving things. Code Bless Ship Maintain Profit! git rm
- 24. Wrap Up • Safety unleashes creativity • Many touch points, many ways to do it — find what’s right for you • It’s not all about technology, most is about the culture and the people