Experience vs Data: A Case for More Data-informed Retrospective Activities
0 likes19 views
The document discusses integrating project data into agile retrospective activities to enhance decision-making based on evidence. It analyzes 140 activities from the Retromat tool, focusing on how data is collected and utilized in retrospectives. The findings reveal that most activities rely on team perceptions rather than project data, suggesting a need for more data-informed practices in retrospectives.
![Retrospective Meetings
3
Definition
opportunity for the team to inspect itself
– The Scrum Guide [Schwaber, 2017]
“ ”
[Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” (2017)
[online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Image: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland
■ Time and space to discuss and improve development process
■ Looking back after a development iteration](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-3-320.jpg)
![5
Motivation
A Case for More Data-informed Retrospective Activities
■ Decisions based on easily collectible team perceptions
■ Modern development practices (& automated tools) allow
insights into teams’ processes via project data [Zaitsev, 2020]
□ version control (what was changed, why, when?)
□ communication tools (what are others working on?)
□ software builds (what is the testing status?)
[Zaitsev, 2020] Anna Zaitsev, Uri Gal, and Barney Tan. “Coordination artifacts in Agile Software Development”. In:
Information and Organization 30.2 (June 2020), p. 100288. issn: 14717727. doi: 10.1016/j.infoandorg.2020.100288.](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-5-320.jpg)
![Retrospective Activities
6
Agendas that Structure Retrospectives
■ Proposed meeting agendas to structure Retros [Jovanovic, 2015]
■ Generalized phases: set the stage, gather data, generate insight,
decide what to do, close [Derby, 2006]
■ Multiple collection efforts [Loeffler, 2017], most comprehensive :
The Retromat online tool, retromat.org [Baldauf, 2018]
[Jovanovic, 2015] Jovanovic, M., Mesquida, A.L., Mas, A.: Process improvement with retrospective gaming in agile
software development. In: Systems, Software and Services Process Improvement. pp. 287–294. Springer (2015)
[Derby, 2006] Derby, E., Larsen, D.: Agile Retrospectives: Making Good Teams Great. Pragmatic Bookshelf Series,
Pragmatic Bookshelf (2006)
[Loeffler, 2017] Marc Loeffler. Improving Agile Retrospectives: Helping Teams Become More Efficient. Addison-Wesley
Professional, 2017, p. 270. isbn: 978-0134678344.
[Baldauf, 2018] Corinna Baldauf. Retromat - Run great agile retrospectives! Leanpub.com, 2018, p. 239.](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-6-320.jpg)
![Activity Extraction
9
Review of Retromat Activities for the Five Retrospective Phases
■ Extracted 35 activities for gather data phase
■ Reviewed all other phases for mentions of “data gathering”
□ Data collection and analysis steps often intertwined
□ Activity's main focus broader than data collection
□ Result: 4 additional activities [1]
e.g. Snow Mountain (using Scrum burndown chart)
[1] https://retromat.org/en/?id=70-84-106-118](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-9-320.jpg)
![Activities Reliant on Data
13
■ Overall nine activities with (possible) project data connections
■ Five depend on interpretation and project context
■ Four make direct mentions of specific development artifacts
□ Tiny fraction of 140 Retromat activities
□ Use data produced in regular tasks of modern software devs
□ Collectable with minimal overhead [Ortu, 2015]
Retromat Activities That Take Project Data Into Account
[Ortu, 2015] Marco Ortu et al. “The JIRA Repository Dataset”. In: Proceedings of the 11th International Conference on Predictive Models
and Data Analytics in Software Engineering. ACM Press, 2015, pp. 1–4. isbn: 9781450337151. doi: 10.1145/2810146.2810147.](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-13-320.jpg)
![Possibly Data-informed
15
Example: Agile Self-Assessment [SeAs]
■ Assessments on team state, based on a predefined checklist
■ Depends on checklist, e.g. “minimal time from pushing code to test”
or “we deliver what the business needs most”
■ Modification: Use checklist featuring measurements based on
Agile practice usage and project data [Matthies, 2016]
Activities That May Use Project Data, Depending on Context
[SeAs] https://retromat.org/en/?id=35
[Matthies, 2016] Christoph Matthies et al. “Agile metrics for a university software engineering course”. In: 2016 IEEE Frontiers in Education
Conference. IEEE, 2016, pp. 1–5. isbn: 978-1-5090-1790-4. doi: 10.1109/FIE.2016.7757684.](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-15-320.jpg)
![Conclusions
16
■ Perceptions vary strongly between team members [Derby, 2006]
■ Project data analyses in Retrospectives enable additional view &
“evidence-based decision making” [Fitzgerald, 2014]
■ Initial Retrospective activities based on data are already exist
■ Team members may focus on interpreting existing data
instead of reconstructing details of the last iteration
Take-Away Messages
[Derby, 2006] Esther Derby and Diana Larsen. Agile retrospectives: Making Good Teams Great. Pragmatic Bookshelf, 2006, p. 200. isbn: 0-9776166-4-9
[Fitzgerald, 2014] Brian Fitzgerald, Mariusz Musia l, and Klaas-Jan Stol. “Evidence-based decision making in lean software project management”. In:
Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014. ACM Press, 2014, pp. 93–102.](https://image.slidesharecdn.com/lasd21-230412131435-963ef158/85/Experience-vs-Data-A-Case-for-More-Data-informed-Retrospective-Activities-16-320.jpg)
Experience vs Data: A Case for More Data-informed Retrospective Activities
- 1. Hasso Plattner Institute University of Potsdam, Germany christoph.matthies@hpi.de @chrisma0 Experience vs Data: A Case for More Data-informed Retrospective Activities Christoph Matthies, Franziska Dobrigkeit virtual conference, January ’21
- 2. Scrum Development Method 2 Iterative Process Improvement: The Retrospective Meeting
- 3. Retrospective Meetings 3 Definition opportunity for the team to inspect itself – The Scrum Guide [Schwaber, 2017] “ ” [Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” (2017) [online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf Image: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland ■ Time and space to discuss and improve development process ■ Looking back after a development iteration
- 4. 4 Team Retrospectives Two separate (?) worlds: Perceptions & Project Data
- 5. 5 Motivation A Case for More Data-informed Retrospective Activities ■ Decisions based on easily collectible team perceptions ■ Modern development practices (& automated tools) allow insights into teams’ processes via project data [Zaitsev, 2020] □ version control (what was changed, why, when?) □ communication tools (what are others working on?) □ software builds (what is the testing status?) [Zaitsev, 2020] Anna Zaitsev, Uri Gal, and Barney Tan. “Coordination artifacts in Agile Software Development”. In: Information and Organization 30.2 (June 2020), p. 100288. issn: 14717727. doi: 10.1016/j.infoandorg.2020.100288.
- 6. Retrospective Activities 6 Agendas that Structure Retrospectives ■ Proposed meeting agendas to structure Retros [Jovanovic, 2015] ■ Generalized phases: set the stage, gather data, generate insight, decide what to do, close [Derby, 2006] ■ Multiple collection efforts [Loeffler, 2017], most comprehensive : The Retromat online tool, retromat.org [Baldauf, 2018] [Jovanovic, 2015] Jovanovic, M., Mesquida, A.L., Mas, A.: Process improvement with retrospective gaming in agile software development. In: Systems, Software and Services Process Improvement. pp. 287–294. Springer (2015) [Derby, 2006] Derby, E., Larsen, D.: Agile Retrospectives: Making Good Teams Great. Pragmatic Bookshelf Series, Pragmatic Bookshelf (2006) [Loeffler, 2017] Marc Loeffler. Improving Agile Retrospectives: Helping Teams Become More Efficient. Addison-Wesley Professional, 2017, p. 270. isbn: 978-0134678344. [Baldauf, 2018] Corinna Baldauf. Retromat - Run great agile retrospectives! Leanpub.com, 2018, p. 239.
- 7. Research Goals ■ Focus: integration of project data sources into Agile Retrospectives ■ Analysis of 140 Retromat activities ■ Review types of data being employed to identify possible improvements. ■ Highlight activities already relying on software project data and those that can be augmented 7 Research Questions
- 8. Research Steps 8 Review of Retrospective Activities and Project Data Usage 1. Extract activities from Retromat that provide or generate inputs for subsequent use 2. Review descriptions, identify data points being collected 3. Categorize data sources 4. Analyze in detail activities which already (or are close to) taking project data into account
- 9. Activity Extraction 9 Review of Retromat Activities for the Five Retrospective Phases ■ Extracted 35 activities for gather data phase ■ Reviewed all other phases for mentions of “data gathering” □ Data collection and analysis steps often intertwined □ Activity's main focus broader than data collection □ Result: 4 additional activities [1] e.g. Snow Mountain (using Scrum burndown chart) [1] https://retromat.org/en/?id=70-84-106-118
- 10. Retrospective Activity Inputs 10 ■ Analyzed textual descriptions of activities ■ Manually tagged with the specific data points collected as inputs ■ Generalized physical representations, index cards ➔ notes ■ Examples: □ Numerical ratings of performed meetings □ Notes on what team members wish the team would learn The Kinds of Data That Teams Work With During an Activity
- 11. Data Source Classification 11 ■ Distinguish whether gathered data is □ drawn solely from perceptions □ extracted from project data □ is ambiguous and depends on project context Analyzing the Types of Data That Is Being Used
- 12. Data Source Classification 12 ■ 86% (30 of 35) of gather data activities do not mention project data ■ Often discussion prompts, e.g. mad, sad, glad □ By default use perceptions and experiences Analyzing the Types of Data That Is Being Used
- 13. Activities Reliant on Data 13 ■ Overall nine activities with (possible) project data connections ■ Five depend on interpretation and project context ■ Four make direct mentions of specific development artifacts □ Tiny fraction of 140 Retromat activities □ Use data produced in regular tasks of modern software devs □ Collectable with minimal overhead [Ortu, 2015] Retromat Activities That Take Project Data Into Account [Ortu, 2015] Marco Ortu et al. “The JIRA Repository Dataset”. In: Proceedings of the 11th International Conference on Predictive Models and Data Analytics in Software Engineering. ACM Press, 2015, pp. 1–4. isbn: 9781450337151. doi: 10.1145/2810146.2810147.
- 14. Activities Reliant on Data 14 The Four Activities That Already Explicitly Rely on Project Data
- 15. Possibly Data-informed 15 Example: Agile Self-Assessment [SeAs] ■ Assessments on team state, based on a predefined checklist ■ Depends on checklist, e.g. “minimal time from pushing code to test” or “we deliver what the business needs most” ■ Modification: Use checklist featuring measurements based on Agile practice usage and project data [Matthies, 2016] Activities That May Use Project Data, Depending on Context [SeAs] https://retromat.org/en/?id=35 [Matthies, 2016] Christoph Matthies et al. “Agile metrics for a university software engineering course”. In: 2016 IEEE Frontiers in Education Conference. IEEE, 2016, pp. 1–5. isbn: 978-1-5090-1790-4. doi: 10.1109/FIE.2016.7757684.
- 16. Conclusions 16 ■ Perceptions vary strongly between team members [Derby, 2006] ■ Project data analyses in Retrospectives enable additional view & “evidence-based decision making” [Fitzgerald, 2014] ■ Initial Retrospective activities based on data are already exist ■ Team members may focus on interpreting existing data instead of reconstructing details of the last iteration Take-Away Messages [Derby, 2006] Esther Derby and Diana Larsen. Agile retrospectives: Making Good Teams Great. Pragmatic Bookshelf, 2006, p. 200. isbn: 0-9776166-4-9 [Fitzgerald, 2014] Brian Fitzgerald, Mariusz Musia l, and Klaas-Jan Stol. “Evidence-based decision making in lean software project management”. In: Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014. ACM Press, 2014, pp. 93–102.
- 17. Future Work 17 The Research Questions and Ideas That Remain Open ■ Extend the toolbox of activities with more data-informed ones ■ Create & evaluate new data-informed Retrospective activities based exclusively on (analysis/visualizations) of project data □ Mining Software Repositories □ Agile Process Measurements Image: https://erikbern.com/2016/12/05/the-half-life-of-code.html
- 18. Summary 18 christoph.matthies@hpi.de @chrisma0 HPI, University of Potsdam, Germany
- 19. ■ sprint retrospective by Eucalyp from the Noun Project (CC-BY 3.0) ■ Meeting by Shocho from the Noun Project (CC-BY 3.0) ■ tools by Nhor from the Noun Project (CC-BY 3.0) ■ Games by Icons Producer from the Noun Project (CC-BY 3.0) ■ Meeting by Chanut is Industries from the Noun Project (CC-BY 3.0) ■ Target by Arthur Shlain from the Noun Project (CC-BY 3.0) ■ Research by Eucalyp from the Noun Project (CC-BY 3.0) ■ Books by sandra from the Noun Project (CC-BY 3.0) ■ Inputs by jngll from the Noun Project (CC-BY 3.0) ■ data classification by Chanut is Industries from the Noun Project (CC-BY 3.0) ■ Data by priyanka from the Noun Project (CC-BY 3.0) ■ Checklist by unlimicon from the Noun Project (CC-BY 3.0) ■ conclusion by Justin Blake from the Noun Project (CC-BY 3.0) Image Credits 19 In order of appearance