Technical stories v1.2
Download as PPTX, PDF6 likes3,231 views
This document discusses technical stories and architecture work in agile software development. It defines technical stories as stories that focus on technical goals rather than user goals. It discusses why technical stories are important for managing technical debt, architecture, and educating the technical team. It provides examples of technical stories and best practices for writing, accepting, and tracking them. The document also discusses architecture, why it is important, and how to plan architecture work through technical stories, architectural runways, and feedback loops.
Technical stories v1.2
- 1. Technical Stories V 1.2 Jim Brisson – Brisson.jim@gmail.com
- 2. Agenda • What are technical stories – User stories vs Technical stories – Technical Story examples – Why are we not creating Technical Stories – What makes a good User/Technical Story – Acceptance and Demo – Where to technical stories come from – When NOT to write technical stories • Tech Debt/System Hygiene – Lehmans’s Law – System Hygiene – Common causes of Technical Debt – Why worry about Technical Debt – Technical Debt can be intentional • Architecture – What is Architecture – Systems are growing rapidly – Why do we need Architecture – Don’t over engineer – Can’t we just refactor? – Use Informed Anticipation – Architecture Runway – Technical Stories on Story Map – YAGNI vs BUFD – Architecture work as systems grow – Feedback loops – Architecture Opportunities – The architecture is in the code – User vs Technical Stories – Architecture Workshops – What? – Architecture Workshops – When and Who? • Green Field – SAFe – LSS • Processing Technical Stories – How to address Technical Stories – Track Technical Backlog with Kanban – Communicating Technical Stories – Team Allocation • Learning – Tech Community of Practice – Learning – Good Architect/Tech Lead – The wrong approach • Summary • Backup
- 3. Technical Stories • How we can – Manage Technical Debt – Manage Architecture – Educate and mentor our technical team
- 5. User Stories vs Technical Stories User stories….. …. highlight the role of the user …. the goal they are trying to achieve …. the value of that goal As a <user type> I want to <achieve a goal> So that I can <get some value> Technical stories….. no user …. the goal you are trying to achieve …. the value of that goal I want to <achieve a goal> So that <some value>
- 6. Technical Story Examples I want to refactor the xyz subsystem using abc pattern to improve robustness. I want to create an enterprise service bus for communication between components to reduce coupling.
- 7. Why are we not addressing Technical Stories? Not clear who is responsible? Not part of the standard workflow? Product Owner doesn’t see the need? Don’t know how to architect/refactor/fix
- 8. What makes a good User (and Technical) Story INVEST • Independent – No dependencies • Negotiable – Does the team have other ideas to achieve the goal? • Valuable – Provides something to the customer • Estimable – Team can understand and approximate the effort • Small – designed, coded, and tested in a sprint • Testable – Good acceptance criteria leads to good test cases
- 9. Technical Story - Acceptance and Demo • Acceptance criteria is still needed – Define success – Define how you will determine the story is complete • You should still Demo the results of the technical story. Examples – Show before and after of refactoring – Review performance measurements spreadsheet (and discuss decisions on next steps) – Review key aspects (data that supports making a decision) for tools researched – Demonstrate how new scripts improve the development process
- 10. Where do technical stories come from • A metaphor referring to the eventual consequences of poor system design, software architecture or software development within a codebase Technical Debt • Areas of code that are difficult to change after implemented • Code that does NOT itself provide user function • Only includes elements understood by all developers Architecture • Research into “how” Spikes • Experiments with “how” Technical Prototypes
- 11. When NOT to write technical stories • For User Story work – Create tasks or acceptance criteria – Non functional requirements may be acceptance criteria on User Stories or may be cross cutting technical story
- 13. Lehman’s Laws of Software Evolution (1980) • Architecture/code degenerates over time • “As a system evolves, its complexity increases unless work is done to maintain or reduce it” • We only understand how the code should be written after it’s done (the first time) Geek And Poke – Oliver Widder
- 14. System Hygiene Technical debt - coined by Ward Cunningham The obligation that a software organization incurs when it chooses a design or construction approach that’s expedient in the short term but that increases complexity and is most costly in the long term What happens if you don’t pay your credit card debt? Touch your ear mini exercise
- 15. But wait • “we don’t have time… the PO bla bla bla, our release date bla bla” • Spare me … • Just Stop It.
- 16. Common causes of technical debt •leads to lack of knowledge Lack of collaboration •work in same branches that will need to be merged Parallel development •putting more code into areas that need refactoring piles on more debt Delayed Refactoring •developer didn’t know better way to write code Lack of knowledge Lack of test suite • pushing to release sooner, rather than completing all necessary changes/tests … quick and messy is ok Business pressures • business decisions are made that ignore the consequences of building technical debt Lack of process or understanding • partially implemented architecture changes, partial adoption of new featurestechniques/patterns Drift • inflexible software that’s difficult to adapt to changing needs Tightly coupled components
- 17. Smells that more refactoring is needed 1. Team is taking longer to deliver features 2. Quality is on a downward trend 3. Team is increasingly frustrated 4. Team won’t touch certain areas of code 5. Team struggles more with estimates 6. Team wants to start fresh, rather than work with current code 7. Team wants more refactoring time 8. Debugging takes longer 9. Fixes or features are implemented through the same change in multiple places
- 18. Why worry about Technical Debt • Future development costs increase as technical debt increases – so less new function can be produced as technical debt rises. • Ignored Technical Debt continues to grow – more and more code is interwoven in smelly areas Productivity Time Clean up old crap
- 19. Why worry about Technical Debt • Debt limits how fully you can develop the Product Function Time IntroductoryStage GrowthStage MaturityStage DeclineStage High Technical Debt Product never reaches full potential Low Technical Debt Product
- 20. Cost of Change - Jim Highsmith • Increased Technical Debt can greatly increase the cost of change • To the point where the system is umanageable “Managing Technical Debt helps ensure reliable delivery today and ready adaptation to tomorrow’s customer needs.” – Jim Highsmith
- 21. Technical Debt can be intentional • Technical debt may result from prudent decisions – e.g. ignoring some finer design points so as to meet an important deadline • No Design may be appropriate to get to market fast, even though more debt is incurred CumulativeFunctionality Time Good Design No Design Down here it may be worth trading off design quality for time to market ... … but up here, there’s no useful trade-off
- 22. Good Technical Debt – Henrik Kniberg New feature • We don’t know best approach • Try several ways – see how they work – comment out as you go • To be good debt • Must Clean Up • Add Tests Time TotalTechDebt
- 23. When incur debt • Don’t lose track of the debt • Add stories to the backlog for that debt, as you go
- 24. Exercise • Discuss with your team, then draw a curve of your component’s accumulation of technical debt over time • Next draw a curve of your ideal technical debt over time • How might you control your technical debt?
- 25. Managing Tech Debt – Henrik Kniberg Measure Debt Ceiling/Baseline? • KISS • Subjective measure – 5 = Great, 1 = Crap Time TotalTechDebt Debt Baseline Debt Ceiling Time TotalTechDebt Debt Baseline Debt Ceiling Quality < 3 Quality = 4 Quality = 5 Time TotalTechDebt Debt Baseline Debt Ceiling Accumulating Debt Cleanup Time TotalTechDebt Debt Baseline
- 26. Architecture
- 27. What is Architecture RUP - “the highest level concept of a system in its environment. The architecture of a software system (at a given point in time) is its organization or structure of significant components interacting through interfaces, those components being composed of successively smaller components and interfaces.” Or – “Architecture is the set of design decisions that must be made early in a project.” Or – “Architecture is about the important stuff.” Or– Customers have a different view than developers, so perhaps Architecture is the highest level concept of expert developers. IEEE – “Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution” Geek And Poke – Oliver Widder
- 28. What is Architecture Francis Ford Coppola – “The key to making a great movie is getting everyone to make the same movie” Ralph Johnson - 'In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called "architecture." This understanding includes how the system is divided into components and how the components interact through interfaces.' Simple definition Architecture: • is about areas of code that are difficult to change after implemented • does NOT itself provide user function • only includes elements understood by all developers Ralph Johnson – “Architecture is shared understanding”
- 29. Systems are growing rapidly Growth of Total lines of code for ~5K Open Source projects (Jun 1994 to Feb 2008) Total software is now estimated to double every two years
- 30. Why do we need Architecture To avoid large scale/unnecessary rework To ensure overall efficiency of Development To minimize impact on deployed systems/users So the code is easier to maintain, easier to extend, is more stable, more comprehensible … Geek And Poke – Oliver Widder
- 31. Where’s the architect • The code is the architecture, and so coders are the architects ….. And the architect codes • Conformance to an architecture plan is determined at the code review and via demos • Nonetheless - Architects whose responsibility spans teams/components can often best address cross component architecture (and process) issues, new tools and new technology
- 32. Don’t over engineer • Challenge - ensure that desirable emergent behaviors result instead of the normal chaos of large organizations • Trying to control this apparently chaotic environment with traditional architectural artifacts, processes, and procedures will only slow the organization down, thus hampering innovation. Andrew Binstock – “Agile does not account sufficiently for architecture and only minimally understands design — two practices crucial to enterprises. Because of these unaddressed needs, enterprises are better off weaving Agile practices with other familiar methodologies and finding a blend that works for them rather than transforming their processes to adopt an approach that, while commendably good at many tasks, remains distressingly incomplete Adrian Cockroft – Thought Leader at Netflix - “Our architecture was changing faster than you can draw it” ,,, ‘Netflix never considered their architecture to be a prerequisite to building software’
- 33. Can’t we just refactor? • Nope • TDD has been claimed to be a design technique … leading to simpler designs … and Refactoring a key part of TDD • IEE 2008 study: – TDD and refactoring had local benefit to code – But … coupling and cohesion were not helped – In fact, there was evidence that TDD hurt Architecture • Can’t create Architecture at the last responsible moment: – Leads to undisciplined form – Leaves a trail of poorly informed code as we go. A good book is not a collection of well formed sentences. There are paragraphs, chapters…. Geek And Poke – Oliver Widder
- 34. YAGNI vs BUFD • Balance – Unearth and act on fundamental knowledge early – consciously try to solve architecture early – take a reasonable guess – But allow knowledge to emerge from local, shortsighted actions – Collaborate on YAGNI decisions YAGNI (your aren’t going to need) – don’t add architectural aspects until you’ve proven you need them BUFD (Big up front design) – try to think of everything you’ll need up front, and lock in the design Which do you think is correct?
- 36. Use Informed anticipation 1. Technical Stories should be written and implemented that anticipate the architecture needed by upcoming User Stories 1. Informed –User Stories must exist that show architectural need 2. Anticipation – Must have a plan for when User Stories will be implemented, so we can properly prepare the runway User Story Technical Story
- 37. Architecture Runway 2. Architecture Runway (a SAFe term) – the technical infrastructure (code) should exist so that high priority features can be implemented without excessive, delay inducing, redesign – User Stories should always be built on an appropriate architecture User Story Technical Story Time Architectural Runway • A user story implies architecture changes • The Architect creates a technical story • The Technical story is implemented in anticipation of the user story (Architectural runway).
- 38. Good runways reduce risk • Consistently completing 100% of your work in an iteration is a bad smell – indicating the team is working conservatively, building in contingency • When good teams badly miss completion targets, often Architecture impacts were misunderstood or missed • Having a good runway – completing technical stories prior to working on the user stories – reduces risk
- 39. How long should the runway be? • Small systems need less runway – If initially wrong, the team can refactor. – Likely is cheaper than trying to anticipate all needs • Larger systems need more – Larger requirements may affect multiple components requiring more widespread architectural changes – Investing in more foresight … and more planning …. can reduce risk
- 41. Architecture Roadmap Opportunities • Mutable • Visible to stakeholders • Integrated into development process • Executed incrementally • Leads to epics/features/stories What: • Big issues/impediments • New tools and technology Can address:
- 42. Big Picture Backlog Item Technical Debt Story Architecture Story User Story StoryFeatureEpic Business Epic Architecture Epic Business Feature Architecture Feature System Test Acceptance Test Unit Test System Qualities Test Nonfunctional Requirement Is One Of Is One Of Is One Of Done When Passes Constrained by Compliant when passes Realized by Realized by
- 43. Theme Theme Feature Feature Architecture Sprint 13 Sprint 14 Sprint 15 Backlog Epic Epic Epic Epic Feature Feature Feature Feature Feature Feature Story StoryTech Debt Story Story Story StoryStory Architecture Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Story Rqmt • Story Map Technical and User Stories – Plan Technical Stories vs User Stories to ensure Architectural Runway exists for User Stories Architecture Architecture Tech Debt Tech Debt Architecture Epic
- 44. Implementing Architectural Epics • Pre-Agile: Create a new branch, code it all up, merge it back in (good luck with that) • Agile – Three choices 1. Big but incremental – best choice – the system always runs. The architectural epic is split into releasable stories. 2. Big but not entirely incremental – 1. Stop and rethink. Isn’t there some way to use the first option? 2. Accomplish across sprints, then ship 3. Increases risk, but may be more efficient 3. Really big and NOT incremental – but you must ship anyway 1. Write the code incrementally as above 2. Initially, isolate the code so that it does no harm 3. Evolve the code to implement the architecture
- 45. Architectural Work as Systems Grow Set up foundation Major Features Rewrite ArchitectureWork Time
- 46. User Stories Are not meant to completely document a requirement – difficult to achieve and time consuming to produce Are meant to be evocative – remind us of the full story Backlog Grooming leads to a shared understanding by the team and the PO The User Story records enough to remind us of the shared understanding Technical Stories and Models Models can help lead us to a shared understanding Technical Stories need only remind us of that shared understanding Architecture workshops create/refine technical stories Models and stories do NOT dictate what must be done, but should be an initial direction that can be modified as we learn more Agile – Make it visible
- 47. Agile - Feedback loops • Command and control stifles creativity, is more brittle – Turns off brain power we’ve already paid for • Instead - Use communication and feedback loops to prevent chaos Architecture Opportunities CoP – for discussion / technical stories Team pairing, mentoring, learning, reviews PDM – User Stories/Tech Stories
- 48. The architecture is in the code – Frederick Brooks in “Silver Bullet” 1987- The building metaphor has outlived its usefulness… If, as I believe, the conceptual structures we construct today are too complicated to be accurately specified in advance, and too complex to be built faultlessly, then we must take a radically different approach.. The secret is that it is grown, not built . Harlan Mills proposed that any software system should be grown by incremental development.. Nothing in the past decade has so radically change my own practice, or its effectiveeness … •Large orgs tend to have architecture groups that lose touch with the code/real architecture (powerpoint architects): •Hand-off waste •Disrespect from coders due to disconnect with code. •Leaders out of touch Powerpoint Architects •Not in documents maintained by an architecture team •The architecture improves and degrades day by day with every line of code added. •The real architecture is a mix of good/bad, intentional/unintentional •Like it or not, at some level, all coders are architects The real architecture is in the code
- 50. Architecture Workshops – What? Get to a shared understanding Use face to face communication with visual models to guide conversation Two primary objectives • Education/coaching/mentoring by more senior/knowledgeable engineers • Shared understanding – documented sufficiently only to remind attendees of the shared understanding Commonly held within team, but may be cross team as needed
- 51. Architecture Workshops – When? and Who? • Right before implementing a requirement or • When design decision needed mid development • May include Tech Lead, Developer(s) and Test Just in time • With groomed requirements, clarify design impacts, and approach to be used. May involve whole team, or subset. • For Dependencies – a cross team meeting is likely appropriate – both teams may draw/discuss their impacts or After Backlog Grooming • May spend a few days or a couple of weeks • Followed by more workshops after backlog grooming • Start with “Walking Skeleton” or Before first iteration of larger project • Review and update opportunities with stakeholders – e.g. via town hall Periodically/at Architecture Opportunity Reviews/Workshops/Milestones
- 52. Communicating stories Technical stories and models are NOT standalone documentation They are intentionally lightweight, meant to evoke memories of a shared understanding When communicating to others (e.g. a rep at a joint workshop explaining outcomes to team mates), the shared understanding needs to be explained, with details filled in.
- 53. Green Field fast start - SAFe • Start with one-several teams to implement architecture underpinnings • Teams are like any other Agile team • Credit for working code, not models or designs • It should take no more than a couple of iterations to create/prove architectural elements Start with Architecture • Add Feature teams to initial set of teams Then move to features
- 54. Green Field fast start - LSS Tiger Team • Form a smaller team with the right expertise • Get the project off to a running start Walking Skeleton • Tiger team works closely together to quickly create a Walking Skeleton • Walking Skeleton has main architecture elements in place, but there is no meat on the bones … most user stories are not yet implemented. Expand • After Walking Skeleton is in place (in short time frame) – seed new teams with tiger team members to expand the ability to satisfy user stories
- 56. How to address Technical Stories Backlog • Collect technical stories in the same backlog as the user stories Prioritize • Regularly prioritize technical and user stories Effort • Estimate effort for both technical and user stories
- 57. How to address Technical Stories Define • At or before Release/MVP Planning Refine • In Backlog Grooming and Architecture workshops • For High Priority technical and user stories • Acceptance Criteria • Refine Story Points Product Owner •Involve the PO •Developing a strategy for implementing Technical as well as User Stories Integrate • Continuously integrate technical stories into development process
- 58. Track Technical Backlog via Kanban 58 Product Analysis Ready for Grooming Ready for Dev • Use same Kanban states to manage Technical stories and User Stories. PBI PBI TBI PBI TBI PBI PBI PBI PBI TBI PBI TBI PBI PBI PBI TBI TBI PBI PBI TBI PBI PBI TBI Proposed TBI PBI TBI Solution Analysis PBI PBI PBI
- 59. Team Allocation • There’s an ongoing cost for system hygiene • Might appropriate capacity to technical stories (for System Hygiene) Team Capacity 100 pts Product Capacity 80 pts System Hygiene 20 pts
- 60. Learning
- 61. Learning • Lack of skills, and lack of knowledge will always be an issue • Education/coaching/mentoring is the remedy In workshops, Technical Leaders, et al need to help others understand the problems, solutions and techniques On the team, technical leaders should look for opportunities to Pair Program, teach Everybody learns - Helping others is key for the organization, and educating will always be a valuable skill in a lead – Folks that don’t understand teach you Code Reviews are a great opportunity to teach Technical Community of Practice
- 62. Tech Community of Practice (CoP) Tech CoP • Share problems and successes • Discuss approaches, modeling techniques • Review Architecture Opportunities
- 63. Good Architect/Tech Lead Strong Collaborator Development team Mentor Guide
- 64. The wrong approach Team members aren’t sufficiently skilled to make important decisions Important decisions must be made early So the architect makes all the important decisions
- 65. Good Architect/Tech Lead Good Architects elevate the team competence so that they can take on more complex issues An Architect’s value is inversely proportional to the number of decisions she makes
- 67. Architecture Documentation Workshop • One or two days • Team splits into small groups to work in parallel at whiteboards: – UML-ish designs Domain Model Logical model Component model Physical model Deployment model Glossary Notes See appendix of this presentation • Show and tell walls – learning and fixing as we go • Preserve on a wiki – photos of white boards • Outcome – Architecture/Design documentation – Much learning Supplies • Lots of whiteboards • Lots of fresh markers • Erasers • Lots of sticky flip charts • Roll of painters tape • 1-2 projectors • Wiki • Video cameras – can record “show and tell”
- 68. Summary
- 69. Summary • Tech stories may cover – Tech Debt/system hygiene – Architecture – Spikes
- 70. Summary • We need Technical Stories – Tech Debt – Balance investment in User Stories vs System Hygiene – Architecture – Just enough architecture Balance YAGNI vs BUFD
- 71. Summary – Architecture Workshops • Architectural Workshops can be done – JIT – Periodically after Backlog Grooming – Major workshops for MVP/Release planning – Periodically to review Architectural Opportunities
- 72. Summary - Learning • Learning is an important aspect of Architecture – Coaching, mentoring, teaching is an important aspect of architecture
- 73. Backup
- 74. Basic Models
- 75. Dell - Internal Use - Confidential Modeling advice • Model what’s helpful • Don’t try to cover all aspects of your code • Don’t worry about including all aspects of the model
- 76. Dell - Internal Use - Confidential Domain model The domain is the actual world from which the problem comes.
- 77. Dell - Internal Use - Confidential State Machine
- 78. Dell - Internal Use - Confidential Deployment model A component is a code module. Component diagrams similar to class diagrams Deployment diagrams show the physical configurations of software and hardware. This deployment diagram shows the relationships among software and hardware components involved in real estate transactions
- 79. Dell - Internal Use - Confidential Use Case • Use case diagrams describe what a system does from the standpoint of an external observer. The emphasis is on what a system does not how. • A scenario is an example of what happens when someone interacts with the system. Here is a scenario for a medical clinic. – "A patient calls the clinic to make an appointment for a yearly checkup. The receptionist finds the nearest empty time slot in the appointment book and schedules the appointment for that time slot. " • A use case is a summary of scenarios for a single task or goal
- 80. Dell - Internal Use - Confidential Activity Diagram (Business Process Model) • Essentially a flowchart overlaid on roles/components
- 81. Dell - Internal Use - Confidential Sequence Diagrams • Interaction diagrams that show how objects collaborate
- 82. Dell - Internal Use - Confidential Collaboration Diagrams • Another form of interaction diagram
- 83. Dell - Internal Use - Confidential Package, Class and Object Diagrams • Should be able to generate package and class diagrams Package Diagram Class Diagram Object Diagram
- 84. Removing Architecture? • Ralph Johnson – – Difference between building architecture and software architecture => It is hard to go back and change your basement, though it is possible. – Irreversibility is one of the prime drivers of complexity. – There’s no theoretical reason that anything is hard to change about software. – But we don’t know how to make everything easy to change.. • Martin Fowler I think that one of an architect’s most important tasks is to remove architecture by finding ways to eliminate irreversibility in software designs.
- 86. 1. Partition by subsystem/product/component • Split epics/features so as to work on one component at a time
- 87. 2. System qualities incrementally • Technical stories often address performance, reliability, security, robustness. Often the improvements can be performed incrementally – Improve performance by 10%, by 20%, by 30% … – Support 5,000 users per server, then 7,000 …
- 88. 3. Incremental Functionality • Stage features of the solution over time • Provide wireless software update for thermostats – Enable update of the device boot loader – Enable update for the radio – Enable update of settings
- 89. 4. Build scaffolding first • Sometimes we need scaffolding to get to the next step. It might be throwaway code, but it facilitates progress • Build an API to support a third-party interface. – Mock the data initially – Enable customers/partners to test integrating with the service
- 90. 5. Major/Minor Effort • Productize APIs – First (major) – separate identified APIs from core service, and use them internally – Then (minor) – Refactor APIs to support standards and third part use
- 91. 6. Complex/Simple • Where the architecture is getting increasing complex, imagine the simplest core piece first, adding variations and complexities later • Complex – Productize the APIs • Simple – Build an adapter to expose a single API
- 92. 7. Variations in data • Implement consumer home energy management devices through the new backhaul grid infrastructure • Implement on meter device using backhaul A protocol • Implement on meter device using backhaul B protocol
- 93. 8. Break out a Spike • A spike or a series of spikes may be needed to establish feasibility, effort or perhaps even to decide how to split the stories • Implement a new messaging queue protocol 1. Create a technical prototype (throwaway) using the new protocol 2. Create test loads with performance tests 3. Run performance tests to determine value