Building Bridges: A DevOps Story

Download as PPTX, PDF

1 like654 views

The document outlines a DevOps story by Matt Campbell, focusing on achieving web-scale in AWS and the strategies for success within engineering teams. Key points include setting clear visions, fostering collaboration, conducting premortems, and promoting empathy and mentorship. The narrative emphasizes the importance of understanding individual contributions and adapting to change.

Engineering

Building Bridges: A DevOps Story

1. Building Bridges: A DevOps Story Matt Campbell matthew.campbell@d2l.com @beardedcoder

2. Matt Campbell Engineering Director with D2L Leading project to achieve proper web-scale in AWS Previously lead move to monthly deployments matthew.campbell@d2l.com @beardedcoder

5. Millions of logins daily at peak TBs of aggregate data PBs of aggregate content Clients with Multi- TB DBs

17. Things you can change

18. Things you can influence

19. Things you can neither change nor influence

21. Things you can change Things you can influence

29. Things you can change Things you can influence

30. Set a clear, achievable vision

33. •Experiences •Skills •Beliefs •Motivations

35. Re-org to bring people together

39. MEGADESSSSSKKKKKKKKKKKK!!!!!

40. WHY? WHY? WHY? WHY? WHY?

41. WHY? WHY? WHY? WHY? WHY?

42. HOW? HOW? HOW? HOW? HOW? https://www.kitchensoap.com/2014/11/14/the-infinite-hows-or-the-dangers-of-the-five-whys/

44. MEGADESSSSSKKKKKKKKKKKK!!!!!

47. Everyone has skin in the game

50. Perform a premortem

53. Mentorship

54. Be brave

58. Be more child-like

61. Be empathetic

63. Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand. Norm Kerth

68. You don’t have to turn this into something. It doesn’t have to upset you Marcus Aurelius

70. 1. Set a clear, achievable vision 2. Re-Org to bring people together • Task Forces • Join in on an RCA and go deep • Physically co-locate • Grab lunch • Some people may not be with you when you are done, have those conversations early 3. Everyone has skin in the game • “Do it with me” versus “Do it to me” 4. Premortems • Delegation poker • Have a neutral party run these

71. 5. Mentorship 6. Be brave • Address problems early • Walk and talk • Appreciate that change is hard 7. Be more child-like • Challenge the status quo • Ask “Why?” • “That’s interesting … tell me more about that” • But also pick your battles 8. Be empathetic • Humanize the problem • Retro Prime Diretive • Embrace failure

72. Things you can change Things you can influence

73. Building Bridges: A DevOps Story Matt Campbell matthew.campbell@d2l.com @beardedcoder

Editor's Notes

#2: https://www.soswildlifecontrol.com/wp-content/uploads/2017/01/qtq80-vB3b6f.jpeg
#5: We’re global https://upload.wikimedia.org/wikipedia/commons/0/09/BlankMap-World-v2.png
#6: We’re data intensive <See if you can get stats on logins, amount of data, size of DBs>
#7: When the company was small this worked just fine. Every had access to everything and people worked close (as in physically) together.
#8: As the company grew, we became larger. Becoming larger means you must become more specialized. Without someone guiding the growth of the company carefully you end up with various groups (factions) all trying to solve their own problem (often the same problem) in their own way.
#9: This causes walls to go up between the factions as we attempt to maintain control over our pocket of the solution. We also begin generating sub-cultures with their own dialects that are focused on what is important to them
#10: This leads to animosity between the groups as it seems as simple as “if they only knew the truth, they would work the way I do” https://pbs.twimg.com/media/DDNN8J4WAAElgZp.jpg
#11: In this talk we are going to focus on the fluffy stuff. A lot of DevOps talks about how to improve the tech to make everything smoother, but much of what doesn’t work involves people or processes.
#12: In my past project, I assumed that tech could solve all the problems
#13: However, CD project we still had many gaps between Dev and Ops Although we made huge improvements, much of it was done through back channels
#14: Didn’t cause the org changes we were hoping for, instead created localized pockets of streamlined behavior http://student.sbc.edu.hk/~111112/images/Streamlined%202.jpg
#15: Need to focus on people, process, and tech https://media.licdn.com/mpr/mpr/AAEAAQAAAAAAAAflAAAAJDE0ZjBmY2I5LWU3OGUtNGNkMC04MDVjLWViM2I4OGYxNzI0Mw.jpg
#17: But what do you pick?
#18: But what do you pick?
#19: But what do you pick?
#20: But what do you pick?
#21: But what do you pick?
#22: But what do you pick?
#23: There are no silver bullets
#24: Can’t I just DevOps it?
#25: No. It’s not that simple https://i.ytimg.com/vi/6w0jakILepk/maxresdefault.jpg
#27: However, CD project we still had many gaps between Dev and Ops Although we made huge improvements, much of it was done through back channels
#29: http://blog.crisp.se/wp-content/uploads/2016/05/Misalignment.png
#30: But what do you pick?
#32: https://pbs.twimg.com/media/CJ-wJPwWUAE5JNk.jpg
#33: I’ve heard a lot of people say our vision is “dealing great client experiences”. However that is not clear nor achievable
#34: We all come into things with different Experiences, Skills, Beliefs, and Motivations. Ops will view that goal as “maximize uptime” whereas Dev will view it as “Deliver features fast”
#35: http://static.wixstatic.com/media/88dc79_291d010ff0fa4439aac8be78fe27b33b~mv2.jpg
#36: https://static.deathandtaxesmag.com/uploads/2014/02/S4_E21_High_Five.png
#37: http://electric-cloud.com/wp-content/uploads/use-case-graphic_continuous-delivery.png
#38: ShadowOps - We were unable to bring our success rates to our goal via tech - Brought together the two teams - That month we hit the goal … without code changes to tie it to https://vignette.wikia.nocookie.net/warcommander/images/9/9e/SpecialOps-HeaderLogo-3.png/revision/latest/scale-to-width-down/300?cb=20140627010700
#39: If a permanent re-org isn’t possible, can you assemble a Task Force to dig into a problem that is currently plaguing your org?
#40: If even that isn’t possible, can you hijack the next RCA and bring Ops and Dev together to discuss in it? - Go blameless - Find where the actual problems are - Focus on How not Why - War Rooms are not Task Forces - Talk about NetApp perf issue https://www.thetreecenter.com/wp-content/uploads/tree-roots.jpg
#44: Ideally, you want people to tell you their stories. I like timelining as an approach for recording this https://fogsmoviereviews.files.wordpress.com/2011/11/20111126-170845.jpg
#45: If a re-org is outside of your control, can you physically move desks to sit closer together? If you can’t as a team, can you as an invidiual? http://i.imgur.com/9uO5owC.jpg
#47: Now is as good as anytime to talk about this. Not everyone will make it through this transformation. The early you can have these tough conversations the better it will be for everyone
#48: Do it with me versus do it to me http://adsolutions.yp.com/sites/default/files/styles/widescreen_large/public/iStock_000010606864Large.jpg?itok=o-0QkiQt
#49: Releasing Machinator as a server-prep tool - But we’re freeing you up to do more interesting things?!?!? - Basically telling this person their job is useless
#50: We created an initiative called Automate Joe. We did … and he ended up quitting (but for different reasons)
#51: https://i.kinja-img.com/gawker-media/image/upload/s--ccAitUxJ--/c_scale,fl_progressive,q_80,w_800/svwa9adt5co36kuoy3mh.jpg
#54: http://3.bp.blogspot.com/-_lIIz3N6r_k/VmelbEabuRI/AAAAAAAAA-E/QshK5u3yby0/s1600/07.%2Bpiggyback.png
#55: http://images.mentalfloss.com/sites/default/files/411363.jpg?resize=1100x740
#56: Address negative behavior immediately. Also be brave in addressing when you do not live up to intentions/commitments http://25.media.tumblr.com/a9dc2aaf291beb323a8c9f155b469bf6/tumblr_mvpcge6obA1r9hvapo3_1280.png
#58: https://quotefancy.com/quote/1243810/Grace-Murray-Hopper-Humans-are-allergic-to-change-They-love-to-say-We-ve-always-done-it
#59: Kids have an inate ability to ask “Why”. They also don’t care what others think about them
#60: Good technique is to say “That’s interesting, tell me more about that” http://s2.quickmeme.com/img/9d/9d56d82da6ff8d9d338e4c2918ab18bfee1f5c8372bee1be2d3cfbcced5b57f4.jpg
#61: Pick your battles (“It is safer to deploy slower”). Some things are ready to be won yet. http://media.fakeposters.com/results/2013/02/03/gwmg4o9vcm.jpg
#66: We are all afraid of failing. Help people embrace and learn from failure
#70: While you may not be able to bring down the walls, hopefully you can at least find better ways to work with them
#71: http://i.telegraph.co.uk/multimedia/archive/02209/clifton_2209817k.jpg
#73: So what did you pick?
#74: https://www.soswildlifecontrol.com/wp-content/uploads/2017/01/qtq80-vB3b6f.jpeg

Building Bridges: A DevOps Story

More Related Content

What's hot (17)

Viewers also liked (7)

Similar to Building Bridges: A DevOps Story (20)

More from Matthew Campbell, OCT (9)

Recently uploaded (20)

Building Bridges: A DevOps Story

Editor's Notes