The Road to Data-Informed Agile Development Processes
1 like149 views
The document discusses the potential of data-driven decision-making (DDDM) in agile software development, highlighting that many practitioners do not value data for making decisions despite its importance. It explores various application areas for DDDM, including business and educational data, and emphasizes the need for integrating data-informed practices into agile processes to improve project outcomes. The author presents research questions aimed at enhancing agile curriculum design and implementing data-informed improvements in software development processes.
![Motivation
2
The “unfulfilled” potential of DDDM
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-2-320.jpg)
![Motivation
3
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7
Software practitioners say:
Data is not valued highly for decision-making
The “unfulfilled” potential of DDDM](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-3-320.jpg)
![Motivation
4
[Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”,
20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7
Software practitioners say:
Data is not valued highly for decision-making, but it should be!
The “unfulfilled” potential of DDDM](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-4-320.jpg)
![“Software is eating the world” [Andreessen, 2011]
Data-Driven Decision-Making
6
[Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.201. 2011
Image: Robyn Twomey for The New York Times, https://www.nytimes.com/2011/07/10/magazine/marc-andreessen-on-the-dot-com-bubble.html
Application areas where DDDM is most prominent](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-6-320.jpg)
![■ “Software is eating the world” [Andreessen, 2011]
■ Software Project Data
□ How often do we actually deploy?
□ How often and why do our tests fail?
□ What are bug-inducing commits?
□ How fast do bugs get fixed?
□ How did a management change
effect our work?
Data-Driven Decision-Making
7[Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.2011. 2011
Application areas where DDDM is most prominent](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-7-320.jpg)
![■ “Software is eating the world” [Andreessen, 2011]
■ Software Project Data
□ How often do we actually deploy?
□ How often and why do our tests fail?
□ What are bug-inducing commits?
□ How fast do bugs get fixed?
□ How did a management change
effect our work?
Data-Driven Decision-Making
8
Application Areas
[Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.2011. 2011
Technical
aspects
Process /
human aspect](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-8-320.jpg)
![Data in Software Development
9
Much more than code
■ Software development process: source of valuable data
□ Not only code: multitude of supporting artifacts
□ (Partly) responsible for project success
■ Processes are “inscribed” into software artifacts [de Souza, 2005]
□ E.g. VCS: who, when, what, why, status
□ Also includes evidence of failures [Ziftci & Reardon, 2017]
[de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). “Seeking the Source: Software Source Code as a Social and Technical Artifact”. In
Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, ACM Press.
[Ziftci & Reardon, 2017] Ziftci, C. and Reardon, J. (2017). “Who broke the build? Automatically identifying changes that induce test failures in continuous
integration at google scale”. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in
Practice Track, ICSE-SEIP 2017, pages 113–122.](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-9-320.jpg)
![Research Questions
14
So many questions, so little time!
■ How can we
□ “lint” project data to find process improvement areas? [1]
□ measure success in implementing agile practices? [2]
□ design curricula and exercises for ease of data collection? [3]
□ use project data to assess curriculum goals? [4]
□ integrate data-informed reflection into Agile processes? [5]
[1] C. Matthies, T. Kowark, K. Richly, M. Uflacker, and H. Plattner (2016), “How Surveys, Tutors, and Software Help to Assess Scrum Adoption,” in Proceedings of the 38th International
Conference on Software Engineering Companion (ICSE), ACM, doi:10.1145/2889160.2889182
[2] C. Matthies, T. Kowark, M. Uflacker, and H. Plattner, (2016) “Agile Metrics for a University Software Engineering Course,” in 2016 IEEE Frontiers in Education Conference (FIE), IEEE,
doi:10.1109/FIE.2016.7757684
[3] C. Matthies, A. Treffer, and M. Uflacker, (2017) “Prof. CI: Employing Continuous Integration Services and GitHub Workflows to Teach Test-Driven Development,” in 2017 IEEE Frontiers in
Education Conference (FIE), IEEE, doi:10.1109/FIE.2017.8190589
[4] C. Matthies, R. Teusner, and G. Hesse, (2018) “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts,” in 2018 IEEE Frontiers in Education Conference (FIE),
IEEE, doi:10.1109/FIE.2018.8659205
[5] C. Matthies, (2019) “Feedback in Scrum: Data-informed Retrospectives,” in Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings (ICSE),
IEEE, doi:10.1109/ICSE-Companion.2019.00081](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-14-320.jpg)
![Research Journey
References
[1] “ScrumLint: identifying violations of agile practices using development artifacts” (CHASE’16)
[2] “How surveys, tutors, and software help to assess Scrum adoption in a classroom software engineering project” (ICSE-SEET’16)
[3] “Teaching Agile the Agile Way — Employing Self-Organizing Teams in a University Software Engineering Course” (ASEE Int. Forum’ 16)
[4] “Lightweight collection and storage of software repository data with DataRover” (ASE’16)
[5] “Agile metrics for a university software engineering course” (FIE’16)
[6] “Prof. CI: Employing Continuous Integration Services and Github Workflows to Teach Test-driven Development” (FIE’17)
[7] “Scrum2Kanban: Integrating Kanban and Scrum in a University Software Engineering Capstone Course” (SEEM’18)
[8] “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts” (FIE’18)
[9] “Attitudes, Beliefs, and Development Data Concerning Agile Software Development Practices” (ICSE-SEET’19)
[10] “Should I Bug You? Identifying Domain Experts in Software Projects Using Code Complexity Metrics” (QRS’17)](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-17-320.jpg)
![The Road to ...
20
“Systematic Literature Review” of Self
■ “A monograph is a specialist work of writing on a single subject […],
often by a single author, and usually on a scholarly subject”
■ Goal: Write a monograph, given a set of previous research results
■ Method
□ Label previous results and papers with topics
□ Form topic clusters
□ Exclude marginally relevant topics](https://image.slidesharecdn.com/fallretreat2019-191019103722/85/The-Road-to-Data-Informed-Agile-Development-Processes-20-320.jpg)
The Road to Data-Informed Agile Development Processes
- 1. Hasso Plattner Institute University of Potsdam, Germany christoph.matthies@hpi.de @chrisma0 The Road to Data-Informed Agile Development Processes Christoph Matthies HPI Research School Fall Retreat, October ’19
- 2. Motivation 2 The “unfulfilled” potential of DDDM [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7
- 3. Motivation 3 [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7 Software practitioners say: Data is not valued highly for decision-making The “unfulfilled” potential of DDDM
- 4. Motivation 4 [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019, doi:10.1007/978-3-030-19034-7 Software practitioners say: Data is not valued highly for decision-making, but it should be! The “unfulfilled” potential of DDDM
- 5. Data-Driven Decision-Making 5 Application areas where DDDM is most prominent ■ Contrasted with decisions based on experience or “gut feeling” ■ Business data □ How much of X did we sell in time Y? □ What ads have converted well in the past? □ Where do users click on our online store? ■ Educational data □ What kind of students are attending this school? □ Have students reached the learning goals?
- 6. “Software is eating the world” [Andreessen, 2011] Data-Driven Decision-Making 6 [Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.201. 2011 Image: Robyn Twomey for The New York Times, https://www.nytimes.com/2011/07/10/magazine/marc-andreessen-on-the-dot-com-bubble.html Application areas where DDDM is most prominent
- 7. ■ “Software is eating the world” [Andreessen, 2011] ■ Software Project Data □ How often do we actually deploy? □ How often and why do our tests fail? □ What are bug-inducing commits? □ How fast do bugs get fixed? □ How did a management change effect our work? Data-Driven Decision-Making 7[Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.2011. 2011 Application areas where DDDM is most prominent
- 8. ■ “Software is eating the world” [Andreessen, 2011] ■ Software Project Data □ How often do we actually deploy? □ How often and why do our tests fail? □ What are bug-inducing commits? □ How fast do bugs get fixed? □ How did a management change effect our work? Data-Driven Decision-Making 8 Application Areas [Andreessen, 2011] Andreessen, Marc. "Why software is eating the world." Wall Street Journal 20.2011. 2011 Technical aspects Process / human aspect
- 9. Data in Software Development 9 Much more than code ■ Software development process: source of valuable data □ Not only code: multitude of supporting artifacts □ (Partly) responsible for project success ■ Processes are “inscribed” into software artifacts [de Souza, 2005] □ E.g. VCS: who, when, what, why, status □ Also includes evidence of failures [Ziftci & Reardon, 2017] [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). “Seeking the Source: Software Source Code as a Social and Technical Artifact”. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, ACM Press. [Ziftci & Reardon, 2017] Ziftci, C. and Reardon, J. (2017). “Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale”. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
- 11. Adapting Software Processes 11 Close the feedback loop
- 13. Research Method 13 Access to student software development teams https://hpi.de/plattner/teaching/winter-term-201920/softwaretechnik-ii.html
- 14. Research Questions 14 So many questions, so little time! ■ How can we □ “lint” project data to find process improvement areas? [1] □ measure success in implementing agile practices? [2] □ design curricula and exercises for ease of data collection? [3] □ use project data to assess curriculum goals? [4] □ integrate data-informed reflection into Agile processes? [5] [1] C. Matthies, T. Kowark, K. Richly, M. Uflacker, and H. Plattner (2016), “How Surveys, Tutors, and Software Help to Assess Scrum Adoption,” in Proceedings of the 38th International Conference on Software Engineering Companion (ICSE), ACM, doi:10.1145/2889160.2889182 [2] C. Matthies, T. Kowark, M. Uflacker, and H. Plattner, (2016) “Agile Metrics for a University Software Engineering Course,” in 2016 IEEE Frontiers in Education Conference (FIE), IEEE, doi:10.1109/FIE.2016.7757684 [3] C. Matthies, A. Treffer, and M. Uflacker, (2017) “Prof. CI: Employing Continuous Integration Services and GitHub Workflows to Teach Test-Driven Development,” in 2017 IEEE Frontiers in Education Conference (FIE), IEEE, doi:10.1109/FIE.2017.8190589 [4] C. Matthies, R. Teusner, and G. Hesse, (2018) “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts,” in 2018 IEEE Frontiers in Education Conference (FIE), IEEE, doi:10.1109/FIE.2018.8659205 [5] C. Matthies, (2019) “Feedback in Scrum: Data-informed Retrospectives,” in Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings (ICSE), IEEE, doi:10.1109/ICSE-Companion.2019.00081
- 15. 15
- 16. Research Journey Based on imagined chronological order
- 17. Research Journey References [1] “ScrumLint: identifying violations of agile practices using development artifacts” (CHASE’16) [2] “How surveys, tutors, and software help to assess Scrum adoption in a classroom software engineering project” (ICSE-SEET’16) [3] “Teaching Agile the Agile Way — Employing Self-Organizing Teams in a University Software Engineering Course” (ASEE Int. Forum’ 16) [4] “Lightweight collection and storage of software repository data with DataRover” (ASE’16) [5] “Agile metrics for a university software engineering course” (FIE’16) [6] “Prof. CI: Employing Continuous Integration Services and Github Workflows to Teach Test-driven Development” (FIE’17) [7] “Scrum2Kanban: Integrating Kanban and Scrum in a University Software Engineering Capstone Course” (SEEM’18) [8] “Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts” (FIE’18) [9] “Attitudes, Beliefs, and Development Data Concerning Agile Software Development Practices” (ICSE-SEET’19) [10] “Should I Bug You? Identifying Domain Experts in Software Projects Using Code Complexity Metrics” (QRS’17)
- 18. Research Journey Based on imagined chronological order ■ Research Journey learnings □ Summarized research question and answer for papers ✓ □ Remember what was actually done ✓ □ Find initial connection between work ✓
- 19. Research Journey Based on imagined chronological order ■ Research Journey learnings □ Summarized research question and answer for papers ✓ □ Remember what was actually done ✓ □ Find initial connection between work ✓ □ Use directly as dissertation structure ✗
- 20. The Road to ... 20 “Systematic Literature Review” of Self ■ “A monograph is a specialist work of writing on a single subject […], often by a single author, and usually on a scholarly subject” ■ Goal: Write a monograph, given a set of previous research results ■ Method □ Label previous results and papers with topics □ Form topic clusters □ Exclude marginally relevant topics
- 21. Integration of data-informed improvement into Scrum 21 Dimensions of “Systematic Literature Review” of Self Measurement of specific Agile practices Project data for Agile process curriculum design Research Topic Research Context Research Goal Similar Agile practices in small teams Teams w/ similar processes, different work contexts Professional teams with customized processes ResearchScope Provide team-specific, targeted feedback Improve overall design of development process Enable improvement in self-managing teams Self Literature Review
- 22. ■ Project data measurements and analysis for well-understood Agile practices in student teams □ Actionable insights into process, allow focused feedback ■ Design pattern for evaluating curriculum goals using project data □ Project data collection at scale with low overhead ■ Approaches for integrating data-informed improvement actions into Agile process flow □ Data-informed Retrospectives □ Issues of professional teams Main Contributions 22 Outputs of Research
- 23. 23 Data-Informed Software Development Process Improvement Challenges ■ Not taking on more interesting research □ It’s what we do, but □ “You have enough” ■ Write (short) introduction that provides (enough) context ■ Provide coherent view of related work The Road to DIASDPI
- 24. 24 Data-Informed Software Development Process Improvement The Road to DIASDPI ■ Would you like to read intro and give your feedback? ■ Would you like my feedback? christoph.matthies@hpi.de @chrisma0
- 25. Summary 25
- 26. 26 ■ Road by parkjisun from the Noun Project (CC-BY 3.0) ■ Agile by BomSymbols from the Noun Project (CC-BY 3.0) ■ Data by Alice Design is Industries from the Noun Project (CC-BY 3.0) ■ Research by Eucalyp from the Noun Project (CC-BY 3.0) ■ Management by Gregor Cresnar from the Noun Project (CC-BY 3.0) ■ Result by Turkkub from the Noun Project (CC-BY 3.0) ■ Technical by Aneeque Ahmed from the Noun Project (CC-BY 3.0) ■ Developer by Becris from the Noun Project (CC-BY 3.0) ■ Cog Wheels by Icon Fair from the Noun Project (CC-BY 3.0) ■ GitHub Mark © 2019 GitHub, Inc. from https://github.com/logos ■ Topic by Adrien Coquet from the Noun Project (CC-BY 3.0) ■ Settings by Joe Mortell from the Noun Project (CC-BY 3.0) ■ Goal by Bharat from the Noun Project (CC-BY 3.0) Image Credits In order of appearance