Doing Analytics Right - Designing and Automating Analytics
0 likes170 views
This document discusses the second part of a series on effective analytics design and automation, emphasizing the importance of aligning metrics with organizational goals. It outlines a structured approach to selecting and automating analytics, including the creation of dashboards and data collection methods. The document also illustrates practical examples of using metrics to identify areas for improvement, emphasizing the need for tailored metrics and ongoing evaluation.
Doing Analytics Right - Designing and Automating Analytics
- 1. Doing Analytics Right Part 2 â Designing and Automating Analytics
- 2. This is the second of a series: 1. Selecting Analytics. Murray Cantor, David West. â Aligning the choice of measures with your organizationâs efforts and goals 2. Designing and automating analytics. Murray Cantor, Nicole Bryan. â A straightforward method for finding your analytics solution âą The dashboards, âą the required data, and âą an appropriate choice of analytical techniques and statistics to apply to the data. 3. Building the Analytics Environment. Murray Cantor, Nicole Bryan. â The data solution architecture and stack â How Tasktop can help. 2 http://tasktop.com/webinars
- 3. Look Whose Talking @tasktop âą Nicole Bryan, VP of Product Management, Tasktop â Passionate about improving the experience of how software is delivered â Former Director at Borland Software â nicole.bryan@tasktop.com | @nicolebryan âą Dr Murray Cantor â Senior Consultant, Cutter Consortium â Working to improve our industry with metrics â Former IBM Distinguished Engineer â mcantor@cutter.com | @murraycantor
- 4. Providing some context Created first software lifecycle bus 2011 Global 500 customers 3 OEMs Created Task Management Category 2009 1000+ customers, 3 OEMs De facto ALM integration for developers 2007 1.5M OSS DLs/month, Majority ISVs Defined Software Lifecycle Integration 2013 Emerging ALM discipline, new product category Created first lifecycle data aggregator 2014 Infrastructure for software lifecycle analytics
- 5. SoâŠ. You have Data, then whatâŠ
- 6. ©2015 Murray Cantor Metrics are essential for sense and respond loops to achieve goals When choosing measures consider whether ⹠The measures let you know how whether you are achieving the goals? ⹠You have a way to respond to the measures? 6 Avoid building dashboards just to use the data
- 7. ©2015 Murray Cantor Choosing metrics big picture Agree on goals - Depends on the levels and mixture of work Agree on the how they fit into the loop 1. âHow would we know we are achieving the goalâ 2.â What response we take?â Determine the measures needed to answer the questions - Apply the Einstein test (as simple as possible, but no simpler) Specify the data needed to answer the questions Automate collection and staging of the data 7 Today Later
- 8. ©2015 Murray Cantor Kinds of Development Efforts: What is your mix? 8 1. Low innovation/high certainty ⹠Detailed understanding of the requirements ⹠Well understood code 2. Some innovation/ some uncertainty ⹠Architecture/Design in place ⹠Some discovery required to have confidence in requirements ⹠Some refactoring/evolution of design might be required 3. High innovation/Low Uncertainty ⹠Requirements not fully understood, some experimentation might be required ⹠May be alternatives in choice of technology ⹠No initial design/architecture
- 9. ©2015 Murray Cantor Different Efforts, Different Goals 9 Cost of work items Lead/Cycle times On-time delivery Value Creation
- 10. ©2015 Murray Cantor From Goals to Measures to Data (GQM-ish) 1. Identify a set of corporate, division and project business goals and associated measurement goals. 2. Specify a sense-and-respond loop to steer to the goal. 3. Generate questions based on the goal that if answered: âą Let you know have achieved, are trending to the goal? âą Provide the level of detail necessary to take action â Where is the problem, bottleneck? âą Communicate progress to stakeholders â Summaries, rollups 4. Select or specify data needed to answer the questions in terms of state transitions of the relevant artifacts 5. Study the data to specify the data set and statistic needed to be collected to answer those questions and track process and product conformance to the goals. 6. Develop automated mechanisms for data collection. 7. Collect, validate and analyze the data in real identify patterns to diagnose organization situation and provide suggestions for corrective actions. 8. Analyze the data in a post mortem fashion to assess conformance to the goals and to make recommendations for future improvements. 10
- 11. ©2015 Murray Cantor Why GQM? ïź Operationalizes âbusiness alignmentâ âą Shared âgoalsâ is the essence of business alignment: ïź It is the method for going from client needs to technical specification âą Like going from user stories to software specifications ïź Disciplined practice âą Avoid platitudes like âOperational excellenceâ âą Avoid Common anti pattern: Lots of metrics, ill focused 11 Doing analytics without GQM is like doing software without stories
- 12. ©2015 Murray Cantor Bucket One Example: Goal: Improve Cycle Time ïź The situation: An level 3 support organization needs to meet stakeholder needs to improve its responsiveness ïź Questions: âą How long does it take to complete? âą Where are the bottle necks? â In the process â Which team ïź The artifact is a defect with states âopened, prioritized, tested, shippedâ. ïź We need the durations of times between âopenedâ and âshippedâstates 12
- 13. ©2015 Murray Cantor To feed an analytics tool like Tableau we need this table: 13 We found the durations using the DATEDIFF() function.
- 14. ©2015 Murray Cantor To Visualize the data, use a histogram 14 80% point is about 105 days
- 15. ©2015 Murray Cantor Blue Team 15 Blue team cycle time 80% point is about 99 days For some reason, the severity 3âs has the best cycle time.
- 16. ©2015 Murray Cantor Red Team 16 Red team cycle time 80% point is about 107 days For some reason, the severity 3âs has the best cycle time.
- 17. ©2015 Murray Cantor Artifacts are in the transitions backlog when they are awaiting transition with no assigned resource Transition Backlog Report 17
- 18. ©2015 Murray Cantor Insights and Actions ïź Insights âą Both teams performing comparably: Not obvious skills issue âą Backlogs too large âą The teams seem to be focusing on the easier, not the most critical ïź Actions âą With team investigate reason for backlog size âą Discovered the governance process (decision to update statuses) is overly cumbersome leaving staff free to work elsewhere âą In response, the governance process was: â Streamlined (an approval eliminated) â Automated (less time spent finding e-mails) âą Work with teams to set and track cycle time 80% goal by priority 18
- 19. ©2015 Murray Cantor This is what improvement looks like 19
- 20. ©2015 Murray Cantor Different Efforts, Different Goals 20 Cost of work items Lead/Cycle times On-time delivery Value Creation
- 21. But wait ⊠thereâs more!
- 22. What weâve learned so farâŠ. âą Webinar 1: There is no âone size fits allâ metric nirvana âą Webinar 2: Use GQM to design the metrics that are right for your mix of development And up next ⊠Webinar #3: Itâs all about the execution! Letâs get practical!
- 23. ©2015 Murray Cantor Choosing metrics big picture Agree on goals - Depends on the levels and mixture of work Agree on the how they fit into the loop 1. âHow would we know we are achieving the goalâ 2.â What response we should take?â Determine the measures needed to answer the questions - Apply the Einstein test (as simple as possible, but no simpler) Specify the data needed to answer the questions Automate collection and staging of the data 23 Today Webinar 3
- 24. ©2015 Murray Cantor From Goals to Measures to Data (GQM-ish) 1. Identify a set of corporate, division and project business goals and associated measurement goals. 2. Specify a sense-and-respond loop to steer to the goal. 3. Generate questions based on the goal that if answered: âą Let you know have achieved, are trending to the goal? âą Provide the level of detail necessary to take action â Where is the problem, bottleneck? âą Communicate progress to stakeholders â Summaries, rollups 4. Select or specify data needed to answer the questions in terms of state transitions of the relevant artifacts 5. Study the data to specify the data set and statistic needed to be collected to answer those questions and track process and product conformance to the goals. 6. Develop automated mechanisms for data collection. 7. Collect, validate and analyze the data in real identify patterns to diagnose organization situation and provide suggestions for corrective actions. 8. Analyze the data in a post mortem fashion to assess conformance to the goals and to make recommendations for future improvements. 24
- 25. A phrase used in the telecommunications and technology industries to describe the technologies and processes used to connect the end customer to a communications network. The last mile is often stated in terms of the "last-mile problem", because the end link between consumers and connectivity has proved to be disproportionately expensive to solve. Read more: http://www.investopedia.com/terms/l/lastmile.asp#ixzz3dAdJpzAQ
- 27. Aspiration without execution is useless! No wait ⊠Itâs actually worse than uselessâŠ
- 31. Remember â you want your point tools to stay focused on their domain expertise
- 34. In the last webinar in this series, we will show you how, by using Tasktop Data, you can solve the last mile problem efficiently and painlessly And, yes, there will be a demo ï
- 36. This is the second of a series: 1. Selecting Analytics. Murray Cantor, Dave West. â Aligning the choice of measures with your organizationâs efforts and goals 2. Designing and automating analytics. Murray Cantor, Nicole Bryan. â A straightforward method for finding your analytics solution âą The dashboards, âą the required data, and âą an appropriate choice of analytical techniques and statistics to apply to the data. 3. Building the Analytics Environment. Murray Cantor, Nicole Bryan. â The data solution architecture and stack â How Tasktop can help 36 http://tasktop.com/webinars