Greenfield Effect: Patterns for Effective Disaster Delivery
Download as PPTX, PDF1 like164 views
The document discusses several significant software failures caused by technical issues, emphasizing the critical impact of bad coding practices on safety and finances. Examples include the Therac-25 and Ariane 5 disasters, illustrating the dangers of neglecting technical debt and reusing problematic code. It critiques the culture of rapid development without quality oversight, advocating for clean coding and better practices while highlighting the long-term consequences of software-induced disasters.
![There is more
1980: NORAD reported that the US was under missile attack. The problem was caused
by a faulty circuit, a possibility the reporting software hadn’t taken into account.
2003: The glitch in St. Mary’s Mercy Medical’s patient management system notified
Social Security, insurance companies, and the families, of demises of 8500 living
patients.
2012: Knight Capital Group’s trading algorithms activated legacy system and decided
to buy high and sell low on 150 different stocks. $440 million lost in 30 minutes.
[page 1 of 100]](https://image.slidesharecdn.com/greenfieldeffect-180627093010/85/Greenfield-Effect-Patterns-for-Effective-Disaster-Delivery-16-320.jpg)
Greenfield Effect: Patterns for Effective Disaster Delivery
- 1. Greenfield Effect Patterns for effective disaster delivery
- 3. 1978 - 1985: hole in the ozone layer over Antarctica remained undetected for ~7 years Root cause: data analysis software used to map the ozone layer had treated correct records of extreme values as measurement errors and ignored them.
- 5. I’ma be progrmr! I’m dumb! got a job on brownfield
- 6. I’ma be progrmr! I’m dumb! got a job on brownfield starts understanding the code mother of god!
- 7. I’ma be progrmr! I’m dumb! got a job on brownfield starts understanding the code mother of god! F. this job!! goes to Conference and hears about all the cool new tech
- 8. I’ma be progrmr! I’m dumb! got a job on brownfield starts understanding the code mother of god! F. this job!! goes to Conference and hears about all the cool new tech hired as first engineer at greenfield writes crappy code others will have to maintain YAY!
- 9. Why is it even bad?
- 10. Why is it even bad?
- 11. Why is it even bad? Greenfield Effect
- 12. Why is it even bad? Greenfield Effect Greenfield Effect Greenfield Effect
- 13. Again, why is it bad? ● Lost money ● Technology advancement slowdown ● Work becomes miserable, depression ● Bad opinion about our profession ● Software-induced disaster where people die ● ^^ already happened, but the bodycount will grow
- 14. Therac-25 Because of a side effect of the buggy software powering the device, number of patients received up to 100 times the intended dose, and at least three of them died as a direct result of the radiation overdose. source: royal.pingdom.com
- 15. Ariane 5 The rocket delivering satellite payload self- destructed 37 seconds after launch. The problem was the result of code reuse from the launch system’s predecessor, Ariane 4, which had very different flight conditions from Ariane 5. More than $370 million were lost due to this error. Casting float to integer!!! source: royal.pingdom.com
- 16. There is more 1980: NORAD reported that the US was under missile attack. The problem was caused by a faulty circuit, a possibility the reporting software hadn’t taken into account. 2003: The glitch in St. Mary’s Mercy Medical’s patient management system notified Social Security, insurance companies, and the families, of demises of 8500 living patients. 2012: Knight Capital Group’s trading algorithms activated legacy system and decided to buy high and sell low on 150 different stocks. $440 million lost in 30 minutes. [page 1 of 100]
- 17. There is more 1991: Patriot battery at Dhahran failed to track and intercept a Scud missile due to a software problem in the system's weapons control compute. 28 killed. 2003: blackout across eight US states and Canada affected 50 million people. Race condition in the code caused 256 power plants to go offline. 2011: Due to a mistake in its computer programming, California gave “non-revocable” preference to approximately 450 violent offenders, who were released and did not have to report to a parole officer.
- 18. “If you want to deliver faster, just hire more engineers”
- 19. “If you want to deliver faster, just hire more engineers” -- some idiot PM
- 20. “If you want to deliver faster, just hire more engineers” -- some idiot PM -- and 1000 others
- 21. hack_it -> release_it -> die -- 99% of startups
- 22. hack_it -> release_it -> sell_it -> make money -- Remaining 1%
- 23. Business at scale 101 hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money-> ...
- 24. Business at scale 101 hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it- >release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it- >release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money-> ...
- 25. Business at scale 101 hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it- >release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it- >release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it- >release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money- >hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it- >make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it->sell_it->make_money->hack_it->release_it- >sell_it->make_money->
- 26. 2 years later
- 27. 2 years later
- 28. Worst diseases are the ones that can stay unnoticed for a long time. Also, everybody lies.
- 29. Technical Debt The Wrong Metaphor
- 31. “This sprint we are planning to incur 10% of technical debt.” nobody, ever
- 32. ONE DOES NOT SIMPLY DECIDE TO PRODUCE TECH DEBT
- 33. Tech debt is JUST a metaphor BAD metaphor Monetary debt is fully quantifiable Tech debt is NOT quantifiable.
- 34. We need to start addressing tech debt Why? Cuz the code is in terrible condition How’s that? You wrote it We had no time to write it clean How much time do you need? dunno
- 35. solutions Greenfield version: write clean code and don’t talk about tech-debt Brownfield version: measure the $#!t out of codebase before you open your mouth … and then use the numbers to get the buy in … but don’t take it too seriously
- 36. I need tools lots of tools
- 37. Docker Redis Mongo Couch Azure Lightbend Ribbon Chef VertX TSDB Slack AWS Spring Gradle Serf Selenium Asgard ZeroMQ AppDynamics Elk Hystrix OpenStack Ignite Ansible Splunk Google Cloud Cassandra Mesos Neo4j Gerrit Sonarqube Iron.io Pagerduty Bugsnag Papertrail Sumologic Mixpanel Slack Logstash Mongo Rackspace TeamCity Jenkins Zookeper Spark Hadoop Zuul Kafka Nginx Vault Eclipse Artifactory Riak Packer Terraform Jrebel BigTable Hazelcast Ganglia Puppet RabbitMQ Consul Vagrant Terracotta Crucible Heroku Cloud Foundry Atlas NewRelic Gatling Graphana Flare Keyspace RAMCloud SchemaFree KAI KDI OpenNeptune Qbase AllegroGraph ArangoDB InfiniteGraph Giraph MarkLogic Neo4J OrientDB Virtuoso Actord FoundationDB InfinityDB Lightcloud LMDB Luxio MemcacheDB NMDB Scalaris TokyoTyrant ZeroMQ Kestrel Kubernetes DCOS CloudFoundry GraphQL ReactNative
- 39. Programmer An organism that turns caffeine into software
- 41. Makes you feel absolutely awesome even if the true state of your body is catastrophic.
- 42. Side effects of cocaine intake Permanent damage to blood vessels of heart and brain High blood pressure, leading to heart attacks, strokes Liver, kidney and lung damage Destruction of tissues in nose Respiratory failure if smoked Infectious diseases and abscesses if injected Malnutrition, weight loss Severe tooth decay Auditory and tactile hallucinations Sexual problems, reproductive damage and infertility Disorientation, apathy, confused exhaustion Irritability and mood disturbances Increased frequency of risky behavior Delirium or psychosis Severe depression
- 43. Cocaine Tools ● if your deployment is too complicated to be scripted in bash… ● if your logs are too tangled to be analyzed with grep… ● if your code is too complex to navigate with vim… ● if your infrastructure is too sophisticated to be managed as VMs… ● if your data is too intricate to define its schema… ● if your business logic is too fluid to be constrained with types…
- 44. No tool
- 45. Bad tool
- 46. Good tool
- 47. Solutions ● Prefer not use frameworks ● Rely on core language features ● Be boring with your tools and extra boring with platforms ● Always do buy vs build analysis ● IDE is for coding
- 49. Never completes the task others have to complete it Introduces tech debt others need to pay it off Introduces bugs others have to fix them Never follows up when code is live others have to follow up The Real 10x Developer
- 50. There are no 10x devs. There are only 0.1x devs.
- 51. 0.1 dev /nɔ:t pɔɪnt wʌn dəv/ A developer who works on a team with a “10x developer”
- 52. Solutions ● Standards + knowledge ● Call out bad practices ● Definition of done ● High truck factor ● Mandated mentoring and learning ● Pair programming ● Measure productivity of TEAMS not INDIVIDUALS
- 53. How to become a 9x developer ● Pareto rule: 90% of the work takes 10% of the time ● Identify what constitutes the 90% of work ● Work only on that ^^ ● Let others work on the remaining 10% ● You will be 9 times more productive than others ● I don’t know how to become 10x :(
- 54. Todo: global problem Like global warming Its everyones problem To fix it everyone has to work together Nobody has incentive to work on it individually
- 55. The Good Reader Or the Evil of Trying to Help
- 58. A Good Reader ● doesn’t care what is read ● doesn’t care about the intent of the code ● cares only about the effect of the code ● covers up for Bad Writer ● prevents tech debt from emerging ● makes everyone think ths s#!t is simple
- 59. Good Listener Analogy ● You are at the meeting ● Someone uses heavy jargon you don’t understand ● But you do not stop him to explain the terms ● Because nobody else does, so they must know it all ● So you feel a little bit dumb ● But you keep your mouth shut ● You don’t know this but… ● everyone else in the room feels the same way
- 60. solutions Learn to say this in public: “I don’t understand”
- 61. solutions You have the right to refuse reading crap You have the right to rewrite that Be a whistleblower It pays off
- 62. Oh really?
- 63. Dunning Kruger Halo those who don’t know they don’t know
- 64. Unskilled persons suffer illusory superiority, mistakenly assessing their ability to be much higher than it really is.
- 66. Which developer would you hire
- 67. Physical attractiveness gives people a HALO effect whereby others are more likely to trust them and think of them as smarter and more talented. Fancy title, eloquent speaking, years of experience Attractive/hipster look, thick glasses, grey hair, male (WTF!?) Fame, wrote books, spoke at conferences Halo Effect
- 68. Dunning Kruger Effect + Halo Effect = People Who Don’t Know What They’re Doing And Everyone Listens to Them
- 70. “I’ve coded this system with my bare hands. I’ve been in the business longer than you live. How can you tell me how to do programming!?”
- 71. “I’ve coded this system with my bare hands. I’ve been in the business longer than you live. How can you tell me how to do programming!?” --VP of Engineering --Chief Technology Officer --Principal Architect --That person whose beard is longer than yours
- 72. Why is it bad? ● Bad decisions picked over good ones ● Everyone has best intentions, this still happens ● No halo - no career #inequality
- 73. solutions there is no concept of mercy in technical discussions
- 75. ● It acquires random companies ● It hires legions of people ● It engages in projects with no ROI ● It creates departments ● It searches for ways to spend ● It doesn’t care about costs When the company is successful
- 76. ● It cuts unneeded positions ● It optimizes processes ● It focuses on projects that bring value ● It dumps unneeded products ● It cares deeply about costs When the company is fighting for survival
- 77. What if tomorrow… we had to cut our engineering by half?
- 78. There is no symmetry scaling up scaling down
- 79. solutions Act like you would if you were broke. Be the Ebendezer Scrooge of software engineering.
- 80. WORD
- 81. Stockholm Syndrome How I learned to love the mess
- 82. “We went through hell to build it. Why would we want to abandon it” --A person held hostage by his project
- 83. Sunk Cost Fallacy Why use third party bug tracking system? We’ve written our own and have a whole team maintaining it. Could use that OS tool for cluster management. Could also use one our company built internally. But we’ll use the one that smart colleague on our team has written two years ago, cuz he’s done so much work on it. We got 10M LOC in C++. We can’t just throw it away and start writing in Java!
- 84. solutions Code is not an asset, it is a liability The more code we maintain, the less we can deliver
- 85. Julian’s Hate Cycle Like Gartner’s, but better
- 88. “All evidence points to OOP being bullshit” “Why OO Sucks” “Ten Things I Hate About Object-Oriented Programming” “Object-Relational Mapping is the Vietnam of Computer Science” “ORM - the Killer of Scalability” “Java Sucks” “Why is Java a bad language?”
- 89. don’t put this stuff in your resume
- 97. #scepticism
- 98. #learning
- 99. #cleanCode
- 100. #understandbusiness
- 101. #scientificmethod
- 103. about me fintech is my battleground LendUp is the best startup spits sarcasm at @hundredmondays We are privileged with choice. Let’s choose work that matters.