Elite Bug Squashing
Download as PPTX, PDF2 likes156 views
The document discusses debugging techniques at different levels from basic to advanced. It begins with level 1 debugging which involves tools like echo and var_dump and strategies like researching issues online and asking peers for help. Level 2 debugging introduces logging, advanced local environments, monitoring and profiling tools. It discusses strategies like replicating issues and eliminating variables. Level 3 involves preventative techniques like defensive programming, code quality tools, and ensuring a learning culture. The document also provides an example case study of randomly occurring cache corruption that was solved through these debugging steps.
Elite Bug Squashing
- 3. Technical Director, Space 48 @Tonegolf71 @Space48ers Tony Brown
- 4. Mage Titans UK – Developer Conference 10th November 2018 @MageTitans Tony Brown
- 5. Cache corruption Randomly occurring, site down
- 6. Random cache corruption • A pre-launch Magento 2 site • Would randomly end up with exceptions • The site would be unavailable until… • …a cache clear would resolve the issue
- 7. Random cache corruption • A pre-launch Magento 2 site • Would randomly end up with exceptions • The site would be unavailable until… • …a cache clear would resolve the issue Where do you begin?
- 8. Level 1 Finding your debugging feet
- 9. Debugging starts on day 1 • Debugging is an integral part of coding • I started with 20% coding, 80% debugging • We should strive for the opposite!
- 10. Types of debugging • Bugs during development • Bugs that are “live” – Feature broken – Performance issues – Site down
- 11. Impact of bugs • The further along the development lifecycle that bugs appear, the greater the impact.
- 12. Level 1 Tooling • Echo / var_dump / print_r … die()
- 13. Level 1 Strategy: Research • The majority of problems you will face have been solved already.
- 14. Level 1 Strategy: Research • The majority of problems you will face have been solved already. • Opensource community rocks! – Stack exchange – Forums – Github – Contribute if you can!
- 15. Level 1 Strategy: Ask your peers • You shouldn’t be fearful of leaning on your team
- 16. Level 1 Strategy: Ask your peers • You shouldn’t be fearful of leaning on your team • Articulating a problem (even to someone non-technical) can result in your brain sparking off new ideas – They may have ideas or even the solution!
- 17. Random cache corruption • Google revealed it to be a config cache issue – But the recommended resolution didn’t apply
- 18. Random cache corruption • Google revealed it to be a config cache issue – But the recommended resolution didn’t apply • No-one in the team had seen the issue before
- 19. Random cache corruption • Google revealed it to be a config cache issue – But the recommended resolution didn’t apply • No-one in the team had seen the issue before • Collaboration did happen, which accelerated progress
- 20. Level 2 Walking the debugging walk
- 21. Level 2 Tooling • Logging
- 22. Level 2 Tooling • Logging Why is the hard drive nearly full?!
- 23. Level 2 Tooling • Logging • Advanced local environments
- 24. Level 2 Tooling • Logging • Advanced local environments • Monitoring - New Relic
- 25. Level 2 Tooling • Logging • Advanced local environments • Monitoring - New Relic • Profiling – Blackfire
- 26. Level 2 Strategy • Can I replicate the issue?
- 27. Level 2 Strategy • Can I replicate the issue? • What has changed?
- 28. Level 2 Strategy • Can I replicate the issue? • What has changed? • Are there any errors?
- 29. Level 2 Strategy • Can I replicate the issue? • What has changed? • Are there any errors? • Eliminate as much as possible at each step
- 30. Level 2 Strategy • Can I replicate the issue? • What has changed? • Are there any errors? • Eliminate as much as possible at each step • Protect through process: – Code Reviews – QA Function – Automated tests where possible
- 31. Random cache corruption • Couldn’t replicate it locally
- 32. Random cache corruption • Couldn’t replicate it locally • What had changed? – Recently gone to multi-server setup
- 34. Multi-server setup Cluster RedisDB Servers Web 1 Web 3Web 2 Admin Load Balancer
- 35. Random cache corruption • Couldn’t replicate it locally • What had changed? – Recently gone to multi-server setup • Eliminate as much as possible – When set to single server, the issue disappeared
- 36. Multi-server setup (restricted) Cluster RedisDB Servers Web 1 Web 3Web 2 Admin Load Balancer
- 37. Random cache corruption • Couldn’t replicate it locally • What had changed? – Recently gone to multi-server setup • Eliminate as much as possible – When set to single server, the issue disappeared • Put logging in place to capture state – Found that the config cache was getting zapped repeatedly
- 38. Level 3 Clear vision and prevention
- 39. Level 3 Tooling • Xdebug
- 40. Level 3 Tooling • Xdebug • Static analysis for quality / best practice – Mess Detector, Copy paste detector
- 41. Level 3 Strategy • Defensive Programming
- 42. Level 3 Strategy • Defensive Programming • Conscious structure of code – SOLID principles
- 43. Level 3 Strategy • Defensive Programming • Conscious structure of code – SOLID principles • Tight feedback loops
- 44. Level 3 Strategy: Culture of learning – Reduce FEAR! • Mistakes and failures are learning opportunities • Ensure people understand they will be supported • Put in place processes to reduce risk and protect people • Actively encourage people to seek improvements
- 45. Random cache corruption • Used Xdebug to find it was a 3rd party module zapping the cache
- 46. Random cache corruption • Used Xdebug to find it was a 3rd party module zapping the cache • Used Redis Monitor to observe the config cache being written to – This revealed that when it was written to with bad data, it always came from the admin IP address
- 47. Random cache corruption • Used Xdebug to find it was a 3rd party module zapping the cache • Used Redis Monitor to observe the config cache being written to – This revealed that when it was written to with bad data, it always came from the admin IP address The old pre-production environment still existed!
- 48. Multi-server setup Cluster RedisDB Servers Web 1 Web 3Web 2 Admin Load Balancer
- 49. Multi-server setup (pre-prod!!) Cluster RedisDB Servers Web 1 Web 3Web 2 Admin Load Balancer Pre-prod
- 51. Bonus Level: The Pomodoro Technique • 25 minute cycles
- 52. Bonus Level: The Pomodoro Technique • 25 minute cycles • At each break, question progress and approach
- 53. Bonus Level: The Pomodoro Technique • 25 minute cycles • At each break, question progress and approach • Encourages focus and reduces chances of going “down the rabbit hole”
- 54. Thank you! • Any Questions? Manchester 10th November 2018 @MageTitans @tonegolf71