Methods used in scaling Scrum: An experience report

Editor's Notes

• #2: This is an experiencereport of a, in my world, prettywellexecutedlarge-scaleScrumin verydemandingcircumstances.To some of you, the practicesdescribedheremaysoundbasicoroldfashioned. Pleaserememberthatfor otherstheymaybe science fiction.Also, bear in mindthattheory is theory, and this is practice.Try and find the partsthataremostinteresting to you.
• #3: About950 peopleworld-wide, about350 in R&D. HQ inHelsinki, 5-6 DCs.Internetsecurity software, backup, sync, othersMillions and millions of end-users
• #4: I willattempt to speak in straightforwardplainlanguage.I willdosomegeneralization andstereotyping to make the messagecrisper. I am alsoattempting to keepyouawake. Somemaygetupset. Thismaybegood as well.Whenever I use a termthatyoudon’tunderstandorotherwisesaysomethingthatsoundsstrange, pleaseask for clarification.I believethatwewillgainmost out of a interactive session, sofeelencouraged to raiseyourhand and interrupt. I willreserve the right to police the situationhowever.
• #5: There is muchmorecontentherethanwehavetimeto cover.IwillexpectthatyouarefamiliarwithScrum. Ifnot, ask the person next to you.Wheneversomethingcomesupthatignitesyourinterest, pleasetell me to digdeeperinto that.Youare in charge of howthistime is spent, soyouneed to governhowmuchtimeweuse on differenttopics. It’salsowise to leavesometime and questions in reserve for the end.By the way. Prettymuchall the picturesarekosher, sofeelfree to askaboutthem as well.Most of the slidescanactuallytake the entirepresentationtime.
• #6: The projectthat I am about to describewasnot a great business success. This is due to the projectpreconditions and environment.Thispresentationfocuses on the projectexecutionthattried to copewith the situation.Itneeds to beunderstoodthatno executioncanmakeup for an impossiblecontext.
• #7: Now, let’s look at how the projectwas..
• #8: The mission was to make a completely new software system, includingclientproducts, backends, business models, architecture, youname it.New technologies: QT, .MSI, backendstuff, Alluser/partnerfunctionalityneeded to beon-parwith the oldone.Manydetails and somesignificantlargerelementswereunclearwhenstartingwork and evenduring the firsthalf a year of the project.Weemployednew planningmethodsas wefelt the existingoneswerenotup to the task (I agree).Weemployedournew R&D process.Wehad a largeamountof new functionalityspecified, and I mean a lot, and allwasmust-have.Wehadabout 7 months for the projectthatactuallytook a couple of yearswithhighlymodifiedscope.
• #9: The new methodsincluded a new companyagileprocessthatwasbasically a slightlymodifiedscaledscrumwithsomeadditionalroles, engineering practicesstandard, and abstactionlevels for differentsizes of backlogitems.
• #10: Allpossiblepotential for problems:DifferentlanguagesDifferentculturesSubcontractor(s)Differentmaturity as organizationDifferentquality of engineeringRadicallydifferenttime-zonesThesehadneverworkedreallytogetherbefore.Integration of a recentaquisitionwasstillunderway.Morethan100 ppl, between8-13 scrumteamsat alltimes.
• #11: Clear:Teamsare in teamrooms, no distributedteamsSubcontractorswereeitheremployed as wholeteamsordistributed to sit in existingteamrooms to join existingteams.Teamsneed to beresponsibleof allconsequences of whattheyproduce Line management separatedfromprojectsteeringseparatedfromrequirementsmanagamentThe organizationwasfocused on projectexecution, to hellwithline management.
• #12: Wetook into use a new backlog management tool(versionone) whichwelaterchanged to another (jira + greenhopper).Thiswasgood, as moread-hocmethodsprobablywouldnothaveworkedwiththatmass.Wealsogotgoodvisibility into progress (real) and howmuchrequirementschanged and howmanywereadded.The poorquality of requirementswasalsovisible to thosewhocouldsee it.Project reportingwasbased on backlog.The toolingwouldhaveenabled a constructivedialogaboutwhat is mostimportant, whythingsaredone, and how to maketrade-offs.
• #13: In order to describesuch a largeamount of work for differentaudiencesweemployed 4 levels of backlogitemsizes:Task (hours)Story (days, fits in sprint)Feature (weeks, fits in release)Epic (visionarycontainer, doesn’tfitanywhere)Later on weadded ’business goal’ or ’roadmapitem’ that is biggerthan a feature butmorerelevant for biz and customersthanepic.Main communicationmethodaboutexpectations, requirements, and allsuchwasface-to-facecommunication, especially in R&D. Thiswasdone to increaseunderstanding and enabledialogue.
• #14: Similarly to a sprintplanningwereallteammemberscooperate to figure out the upcomingsprint, the projectteamsgathered to Business Iterationplanningevery 2-3 months to prepare for the next Business Iteration. Thispractisewasstartedwith the help of Dean Leffingwell. In this 2-3 dayworkingoff-sitemegameetingpeoplewerealigned, dependenciesfigured out, and initialplans made.DeathbypowerpointPlanningstartsHelpdesksbydifferentsupportfunctionsManagement availableIssuemitigationscrum-of-scrumsevery 2 hoursArchitecturemeeting at intervalsMidpointprogressreviewFinalplanreview
• #15: Thereweremanydifferentcombinations of projectsteering/managemententitiesduring the lifespan of the project.As a trendtherewasmuchmore management and control in the beginning and less in the end.
• #16: Inpractice the mostimpactfulprojectsteering and communicationhappened in daily scrum-of-scrumsthathappenedface-to-face as much as possible.To aidtheseweemployedStatus radiatorsTeleconferencingequipmentRules and guidelines on how to run the meetingsRolingEtherpad for multi-usermeetingminutesIn the meetingweGained a view to currentsituationfrommanyanglesSolvedsimpleproblemsquicklyorAgreed to a spinofforAgreedwhowillcontactwhoAsked for and offered helpBroughtupproblems and potentialsolutionsAgreed on actions
• #17: The directionhasbeen to moveresponsibility and authoritydown to the hands-onpeople. Thisshowswellwith the betareleasingdecisionsthatused to be made by PSG butwerelatergiven to SoS.In thispicturetherearevirtually no managers.Youcanalsosee the improvedtelecomequipment.
• #18: Multi-levelintegrationfrom single developer to team to project.Commonlyagreed and iterativelyimproved.Jenkins. One trunk. Lastknowngood. Release fromtrunk.
• #19: Performed in the teamsbyquality engineering specialistsCoordinatedmostlyby the teamsExploratorytestingestablishesqualityestimateTestautomationmonitorsbasicqualityNo systemtestingteam, no externaltestingteam.
• #20: Mandatorypart of large-scaleagileeffort, enablingcontinuousintegration, monitoringquality, iterativeimprovements and so on.Needs to beperformed in multipleabstractionlevels and for differentmaturitybranches of the product. Crucial for maintaining a near-shippablequality.Tooling is notrequired as youdevelop software for a livinganyway. Social mass is moreimportantthantechnicalmass.
• #21: The developmentenvironmentswerenotadvancedenough for the needs of thisproject, so the projectstafftookcharge and built a new generation of developmentenvironmentsthatwerequicklyadopted for globalcompanyuse. Ifyourtoolsareinsufficient, makebetterones.Fromindividualmachines to farm of vmware to dynamicallyprovisionedvirtualmachines in KVM.
• #22: Bugdecisionswerehandledbyhands-onstaff (according to a companyguideline), everyday. Decisionsweredemanded, ignoringbugswasnot an option.Bug countswerestillrising, so a policy of stop-the-linewasinvented (namedafter the famous Toyota practice).Priorities:FixnowfixnextsprintMove to backlogtrash
• #23: Initiallyweaimed to publishbetareleaseseverymonth, butitproved to betoodifficultsowestarteddoingitaftereverytwoweeksprint.Betareleaseswere a majorcontributor to stablequality, integration and internalcommunication. ”TP is believing”.
• #24: Thisprojectsuffered an incredibleamount of disruptions, includingnumerouscompletedirectionchanges, scopechanges, schedulechanges, architecturechanges, staffingchanges, reorganizations, and evenlayoffs. Nevertheless the projectwasable to continuallymakeprogress and keep publishing working software. No meanfeat.
• #25: Nowtime for the reservequestions.
• #26: Someadvice,ifweget to this.