A real-life overview of Agile and Scrum
- 1. Boston Php 9/21/2016 A real-life overview of Scrum @mtoppa Michael Toppa www.toppa.com
- 2. www.toppa.com @mtoppa Mike Toppa web engineer 610-322-7034 mike@toppa.com www.toppa.com Ruby, Rails, PHP, Wordpress, JS, HTML, CSS SQL, NoSQL, AWS, TDD, Scrum, Kanban 20 years of experience in web development, project management, and functional management. I’ve worked in a variety of environments: dot coms, universities, start-ups, consultancies. One thing I’ve learned is that organizations are like families - they’re all dysfunctional - it’s just a question of how badly
- 3. Overview ✤ Tell me your problems… ✤ How traditional software development workflows can go wrong ✤ Tell me what you want… ✤ What makes software development enjoyable and rewarding? ✤ How do we get there? ✤ Agile principles ✤ The Scrum approach ✤ Additional best practices ✤ Scrum adoption anti-patterns and common difficulties
- 4. Tell me your problems…
- 5. Features Cost Schedule 1.The iron triangle Client can pick two Quality I’ve explained the triangle to dozens of clients over the years. Programming is not magic. If the client tries to squeeze all 3 sides of the triangle, quality suffers.
- 6. Misalignment of authority and responsibility Cartoon by Mike Lynch Used with permission - Following this advise lets you cover yourself politically, and is a great way to make everyone who works for you miserable - I've found that misalignment of authority and responsibility can explain a lot of dysfunction that happens in organizations - When you have responsibility for your work but not enough authority over it, you will feel like a cog in machine
- 7. “If you go to the store with a huge shopping list and twenty dollars, you need the authority to go to the money machine for more cash, or the authority to make changes to the list.” Ron Jeffries, Making the Date What’s happening is that the client is trying to retain authority on the project while giving you the responsibility. But ultimately, for the project to be successful and for both you and the client to be happy, responsibility and authority need to be brought into alignment.
- 8. 2. Multiple projects and multitasking Source Context switching between two projects eats about 20% of a full-time worker’s schedule. The sense of progress with multitasking is an illusion, compared to not multitasking
- 9. Source
- 10. My experience at the U Penn School of Medicine
- 11. Before Scrum: web team setup
- 12. With Scrum: web team setup
- 13. Too much work SWAG chart 9 developers, 2 product owners, and me supporting - 22 clients with 124 applications 3 designers and 1 product owner supporting - about 200 static content web sites Taking inventory itself was a huge undertaking
- 14. Source To have any chance of success in the long run, you have to claim authority you may not have had previously. You may have to fight for it…
- 15. Source …but you have to always be professional. Think of how doctors behave in an ER. When the pressure is on is when you want them to be at their most professional.
- 16. 3.The cone of uncertainty Source
- 17. 4. Information is lost in handoffs Source
- 18. A perspective on the traditional approach Source
- 19. Traditional “Waterfall”Approach Source Features determine the cost and schedule Define all requirements up front Logically break down the work Estimate the effort / durations Plan out all the work And only then begin the development…while trying to limit any change that will threaten the plan.
- 20. “I find your lack of faith disturbing” 5. No opportunity for feedback Source In combination with the cone of uncertainty, this is deadly
- 21. Result: we build the wrong features Source Ask customers what they want at the beginning, when they really don’t know Penalize them for adding things later TCL example
- 22. “The main thing that pushed Agile and Scrum was that the success rate on traditional projects was terrible; it was 45%. If that was a car-manufacturing place, that would mean you’d throw out every other car you built.” Ken Schwaber, co-creator of Scrum, 6/21/2011 Overall result: low success rate Source
- 23. Tell me what you want… For yourself, and for your customers
- 24. What makes a job enjoyable? ✤ Autonomy ✤ Reward for effort ✤ Challenging/complex work “Work that fulfills these three criteria is meaningful.” – Malcolm Gladwell, “Outliers: The Story of Success”
- 25. “Novices believe that quality and velocity are inverse. They think that hacking is fast. They haven’t yet recognized what professional developers know all to well: …the higher the quality, the faster you go” Bob Martin, Vehement Mediocrity
- 26. How do we get there?
- 27. Add more people? Brooks’ law: ”Adding manpower to a late software project makes it later” - Fred Brooks, The Mythical Man-Month
- 28. Hold the teams feet to the fire? Source This is the death march. Analogy that software development is like a washing machine.
- 30. image source * In 2001, a group of 17 very experienced software developers got together and came up with the Agile Manifesto. They had spent their careers experiencing all the difficulties of the traditional approach, and were motivated to come up with something better
- 31. “Agile” consists of a set of principles and ideals
- 32. Scrum is the most popular Agile methodology The original ideas for Scrum came from an article published in Japan in the mid 1980s, which reviewed case studies of early implementations of the ideas in Japanese manufacturing work. In the 1990s Ken Schwaber and Jeff Sutherland gradually formalized Scrum into a set of specific practices for software development.
- 33. Agile solution: flip the triangle Source Agile takes into account the “cone of uncertainty” - things will change
- 34. Agile: frequent feedback is key Source Rather than fight the “cone of uncertainty” we embrace it. We are always checking in to make sure what we’re delivering is what the client wants, and we’re ready to adjust priorities based on feedback. At some point we will run out of time or money, and when that time comes, we want to make sure we have delivered the most important features.
- 35. Source
- 36. Source Develop systems in small portions at a time (incremental), through repeated cycles (iterative), and take advantage of what was learned in each cycle (for both features and implementation)
- 37. Incremental development: slice vertically, not horizontally Source This is where developers unfamiliar with Agile freak out How do you develop a UI or a database in pieces? This seems like it would lead to a giant mess. Remember the iterative part - we can sketch out the overall design, but we build incrementally, fleshing out the details of what’s needed soon It is possible, it is practical, and there are a lot of people doing it. It's actually the opposite of messy hacking. Doing it well requires a very disciplined development process, and strong application design skills, as you are trying to maintain a solid application design while always being ready to adapt to change.
- 38. The Agile workflow ✤ Deliver business value incrementally ✤ Deliver in order of business priority ✤ Deliver in small, frequent batches ✤ Solicit and respond to business feedback at each delivery ✤ Measure what is left to do ✤ Make decisions based on track record ✤ Stop when it makes sense Source
- 39. Why? ✤ The pace of business keeps getting faster ✤ Feedback is essential ✤ Time is scarce ✤ Things will change
- 40. Agile:“inspect and adapt” Source Single loop learning is “how can we do better”? Double loop learning is “Why do we believe that?” Double loop learning means challenging fundamental assumptions
- 41. Applying Agile with Scrum
- 42. Scrum: overview ✤ Scrum consists of these core elements: ✤ 3 roles ✤ 3 artifacts ✤ 5 events ✤ That’s it! ✤ But there are also several recommended best practices The scrum book we’ve been reading presents doesn’t really distinguish the core elements from the recommended best practices.
- 43. Scrum has 3 roles Highly collaborative approach - minimize handoffs and maximize advantages of bringing together people with different perspectives and backgrounds
- 44. Scrum role: Product Owner Responsible for what the team will work on, and setting priorities, but not how the work is done * Responsible for what the team will work on, but not how the work is done * Works closely with clients to understand their needs * Gathers and writes business requirements in small pieces, called “user stories” * Based on client needs, sets priorities for the team * Does not have authority over technical design decisions * Cannot tell an individual team member what to do * A good Product Owner is: available, business-savvy, communicative, decisive, empowered
- 45. Scrum role: Scrum Master A “servant-leader” for the team * A “servant-leader” for the team - analogous to a physical trainer * Can coach and evangelize, but not issue commands * But does have authority over the Scrum process * Removes obstacles for the team * A good Scrum Master is: responsible, humble, collaborative, committed, influential, and knowledgeable
- 46. The team: self-organizing and cross-functional Source * Cross-functional team takes collective responsibility for estimating the work, and doing the work * Doing it in the priority order they are asked to follow * Keeping quality high by working together (inspecting each other's code, discussing best technical solutions, etc)
- 47. So who’s the boss? Source Personnel management exists completely outside this structure. Works best in relatively “flat” organizations where people are given autonomy and achievable goals Antithetical to top-down, command and control hierarchies
- 48. Scrum workflow: events & artifacts Source
- 49. Artifact #1: the product backlog Source * The product backlog is the prioritized list of features, usually written in the form of user stories. * It is a living document, subject to re-prioritization between sprints. * The product owner manages it
- 50. Event #1: the sprint (aka increment) Source * A regular, repeating work cycle * Sprints should have a consistent length, no longer than 1 month * Good to start with 2 weeks, and then adjust if needed * Purpose: * Establishing a regular cadence: a sprint should never be extended * Get away from slow periods at the start of a project and death marches at the end * Delivering working software at the end of each sprint becomes routine * As you get better at scrum, you’ll want to move to shorter sprints, but usually no shorter than 1 week
- 51. Artifact #2: the sprint backlog & Event #2: sprint planning Source * In the sprint planning meeting, the team reviews the highest priority stories from the product backlog and breaks them down into tasks. * They estimate how many of the stories will fit into the sprint. * Key point: the team decides how much can fit in the sprint, not the Product Owner. The team has authority over what they are responsible for. * Ideally there can be a “sprint goal” where the set of stories come together as a feature set, but this isn’t always possible.
- 52. Artifact #3: the “product increment” Source To complete a campground project, we can deliver these features, one at a time etc… * To deliver a holiday park, we can deliver these complete features, one at a time: * Campsite for tents * Reception area, for check-in and check-out * Toilets and showers * Hook ups for RVs * Laundry facilities * A shop * Etc…
- 53. Event #3: the daily standup (aka daily scrum) Source * You literally stand * Timeboxed to 15 minutes * The goal is to answer the 3 questions: "What did you do yesterday?", "What will you do today?", and "Are there any impediments preventing you from completing your work? * This meeting is by and for the team, to speak candidly about the status of the work * The goal is to make sure everyone knows what’s going on, and to deal with any impediments quickly * Management is ideally not there. If they need to be there, they do not speak and definitely do not run the meeting. They should stand behind the team, so it’s clear the attention is not on them
- 54. Remote standups at ElectNext * Works for remote teams too * And in small organizations with good working relationships, everyone can join, not just developers
- 55. Event #4:The sprint retrospective Source * Post-mortem meeting at the end of each sprint. * “Inspect & Adapt” is the focus * Length rule of thumb: 1 hour per week in the sprint (2 weeks = 2 hours) * The team generally focuses on these questions: * What went well, and can we learn from it? * What didn’t go well, and what can we learn from it? * In general, how can we do better? * I’ve found it’s also good to review progress on goals from previous retrospectives * Management is absolutely not present, unless there’s a special reason
- 56. Event #5:The sprint review Source * The product owner runs the sprint review * Length rule of thumb: same as retrospective * Demo the work of the sprint to all stakeholders: customers, CEO, etc * Get feedback: inspect and adapt the software
- 58. User stories
- 59. User story with “conditions of acceptance” from a U Penn project
- 60. …and a mockup to go with the story
- 61. Splitting big stories ✤ “A job seeker can post and manage her resume…” ✤ One way to split it is along CRUD lines: ✤ create a resume… ✤ delete a resume… ✤ etc. ✤ Another way is data boundaries: ✤ add and edit education information… ✤ add and edit job history… ✤ etc.
- 62. Good stories: INVEST ✤ Independent ✤ Negotiable ✤ Valuable to customers ✤ Estimatable ✤ Small ✤ Testable
- 63. Estimating: story points Source * People are bad at estimating time, but we're good at estimating relative size or difficulty * As the points go up, the range of uncertainty also goes up
- 64. Estimating: planning poker Photo by Kelly Hirano Used with permission * The teams give “story points” to the work by playing planning poker * After the PO describes the story and answers any questions from the team, each team member shows their card at once. This prevents them influencing each other. * Team based estimates are more accurate than estimates by individuals
- 65. Velocity enables scheduling and “sustainable pace” Source * After a few sprints, teams have velocities, which allows for making time estimates for projects. * And this is key to the Agile goal of “sustainable pace”
- 66. Many more I don’t have time to describe…. ✤ Handling non-functional requirements ✤ Handling back-end system stories ✤ Release planning ✤ Burn down charts ✤ Impact mapping ✤ Specification by example ✤ Pair programming ✤ Automated unit testing and feature testing ✤ Continuous integration ✤ “Walking skeletons” ✤ Etc
- 67. Scrum adoption anti-patterns and common difficulties
- 68. #1: the top-down, lip-service approach Scrum master hiring story
- 69. Source
- 70. #2. Misunderstood and/or misapplied Source
- 71. Source
- 72. A Scrum coach can help Source * A skilled external coach is often key for driving change - they bring a wide range of experience and can see things objectively * If you’ve never led an Agile transition before, it’s surprisingly easy to do it wrong * At Penn, I misunderstood the roles: the clients were acting as their own product owners, The scrum masters were our former project managers, and continued doing traditional project management * You need at least enough management support to pay the coach * You need to make sure you’re bringing in someone good
- 73. #3: Not realizing what you’re getting into
- 74. The Scrum Promise “In my Scrum classes I warn attendees of what I call the Scrum Promise: If you adopt Scrum, there will be a day you come into the office nearly in tears over how hard the change can be. This is because Scrum doesn’t solve problems, it uncovers them and puts them in our face. Then, through hard work we address them.” – Mike Cohn, Agile Trainer A common reaction when this happens is to blame Scrum rather than actually deal with the issues.
- 75. Starting with a pilot project helps
- 76. #3: teams doing both development & operational support
- 77. Theory… Source
- 78. Reality… Source Team didn’t like it, and clients didn’t like it. It felt like just adding a new set of work and processes on top of the existing work
- 79. Handling “emergencies” ✤ Triage… and reject if not really a software issue ✤ Triage… and defer if important but not truly an emergency ✤ Reserve a buffer in the sprint for unplanned work… but beware! ✤ Deal with root causes… prevent ongoing emergencies Source
- 80. #4: Stuck with multiple projects
- 81. Additional references ✤ “Succeeding with Agile: Software Development Using Scrum” and “Agile Estimating and Planning” by Mike Cohn ✤ “Specification by Example” and “Impact Mapping” by Gojko Adzic ✤ “Agile Retrospectives: Making Good Teams Great” by Esther Derby ✤ Angry Dinosaurs: Accelerating Change and Institutional Incompetence presentation by Cory Ondrejka, Wharton Web Conference, 2010 ✤ “The Nature of Software Development” by Ron Jeffries
- 82. References: technical practices ✤ “Clean Code” and “Agile Software Development, Principles, Patterns, and Practices” by Bob Martin ✤ “Refactoring: Improving the Design of Existing Code” by Martin Fowler ✤ “Agile Database Techniques” by Scott Ambler ✤ “Working Effectively with Legacy Code” by Michael Feathers
- 83. Any questions? @mtoppa Michael Toppa www.toppa.com Boston Php 9/21/2016