SlideShare a Scribd company logo

Using Data to Inform Decisions in Agile Software Development

Christoph Matthies, profile picture
Christoph Matthies

The document discusses the significance of data-driven decision-making in agile software development, emphasizing the need to replace anecdotal evidence with empirical data to enhance project success. It highlights the challenges that agile teams face in utilizing available data and suggests that leveraging data can inform critical decisions in software projects. The content references various studies and observations on decision-making processes within agile frameworks, underscoring the importance of feedback and continuous improvement.

Software
Related topics:
agile-software-development • Data-Driven Decision Making • Insights on Software Development • Software Development Process
1 of 39
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
Hasso Plattner Institute University of Potsdam, Germany christoph.matthies@hpi.de @chrisma0 Using Data to Inform Decisions in Agile Software Development Christoph Matthies, Guenter Hesse Prague, July ’19
Motivation 2 for using data in software engineering Software engineering must shed the folkloric advice and anecdotal evidence [...] and replace them with convincing results, established with [...] empirical methods – Bertrand Meyer [Meyer, 2013] “ ”[Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18. Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg
Importance of Decision-Making 3 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
Importance of Decision-Making 4 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis ■ Correct project decisions critical for success [Drury, 2012] □ Overruns of budget and schedule □ Project cancellation [Taherdoost, 2018] [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
Motivation 5 for using data in Agile software development processes Scrum is founded on empirical process control theory [...] knowledge comes from experience and making decisions based on what is known. [...] – The Scrum Guide [Schwaber, 2017] [Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017, [online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland “ ”
Decision-Making in Agile 6 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
Decision-Making in Agile 7 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations Main Research Questions ■ What data is available and relevant to Agile teams? ■ What methods are transferable from other domains? [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
Related Work [Svensson, 2019] 8 Unfulfilled Potential of DDDM in Agile [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
Potential of DDDM 9 Lots of potential, little usage [Svensson, 2019] ■ Survey of software practitioners ■ How is data used in the company for making decisions? [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
Data-Driven Decision-Making 10 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
Data-Driven Decision-Making 11 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies ■ Technology use in business has fueled DDDM adoption □ Increased company productivity [Brynjolfsson, 2011] □ Transparency & real-time information intensifies tensions, management must adapt [Schrage, 2016] [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
Agile Software Development 12 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
Agile Software Development 13 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] ■ Data enables teams to implement Agile feedback cycles ■ Agile, naturalistic decision-making □ Situational behavior, context-dependent rules [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
Levels of Decisions 14 The impact of Agile “self-organizing” teams ■ Strategic plans define long-term goals ■ Tactical plans involve managing projects to achieve goals ■ Operational plans define milestones, and the actual work
Data Use in Business 15 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
Data Use in Business 16 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] ■ Continuing digitization allows additional data collection □ “Reality mining”: cars, smart devices, etc. [Svensson, 2019] □ Clickstream data, UI interactions, high time-resolution [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
Data in Software Development 17 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
Data in Software Development 18 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts ■ Processes are “inscribed” into software artifacts [de Souza, 2005] □ E.g. VCS: who, when, what, why, status □ Also includes evidence of failures [Ziftci & Reardon, 2017] [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
Software Project Data 19 [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
Software Project Data 20 “a gold-mine of actionable information” [Guo et al., 2016] Primary purpose: communication [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.
Software Project Data 21 Use this data for reflection [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
Scrum Process 22 Where Agile decision-making happens [Schwaber & Sutherland, 2017] 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review [Schwaber & Sutherland, 2017] Schwaber, K., & Sutherland, J, “The Scrum Guide - The Definitive Guide to Scrum: The Rules of the Game”, 2017, [online] Available: http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
Sprint Planning 23 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review Building a backlog of work for the next Sprint
Sprint Planning 24 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence
Sprint Planning 25 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence ■ Enable short feedback cycles between devs and real customers □ Metrics from deployed software, real use cases □ Observe actual (mis)usage, engineering relevant metrics □ Base changes on results
Daily Scrum 26 Quick status update 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
Daily Scrum 27 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed
Daily Scrum 28 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed ■ Development artifacts hold information on what was done □ Concrete items that were completed □ Rely less on personal recollection □ Optimize for speed
Sprint Retrospective 29 Reflecting on past work and improving 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
Retrospectives 30 Reflecting on past work and improving What should be improved? Retrospective Meeting
31 Did we improve what we planned? Retrospective Meeting Retrospectives Reflecting on past work and improving
32 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
33 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
Related Research Fields 34 Mining Software Repositories ■ Draw from MSR techniques [Prana, 2018] ■ MSR: Focus on large amounts of repositories □ “most widely used open source license?” [Dyer, 2013] ■ Little research: Few repositories, intricate knowledge of creators / users [Prana, 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of GitHub README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3 [Dyer, 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software Engineering. pp. 422–431. 2013. IEEE.
Conclusion 35 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
Conclusion 36 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
Conclusion 37 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] Data analysis and interpretation by Agile teams (supported by context changes) can enable better informed decision [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
Summary 38
Image Credits 39 In order of appearance ■ Data directed Decision Making by H Alberto Gongora from the Noun Project (CC BY 3.0 US) ■ Project by useiconic.com from the Noun Project (CC BY 3.0 US) ■ Question by unlimicon from the Noun Project (CC BY 3.0 US) ■ Business by David Glöckler from the Noun Project (CC BY 3.0 US) ■ Agile Development by Bê Sabino from the Noun Project (CC BY 3.0 US) ■ Digitization by priyanka from the Noun Project (CC BY 3.0 US) ■ Developer by becris from the Noun Project (CC BY 3.0 US) ■ Plan by Gregor Cresnar from the Noun Project (CC BY 3.0 US) ■ Daily Scrum by Swen-Peter Ekkebus from the Noun Project (CC BY 3.0 US) ■ Documents by LUFTFI GANI AL ACHMAD from the Noun Project (CC BY 3.0 US) ■ Idea Generation by priyanka from the Noun Project (CC BY 3.0 US)

More Related Content

PDF
An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives
Christoph Matthies
 
PDF
Data Science towards the Digital Enterprise
Jake Bouma
 
PDF
Z group project guide
IDES 2105 Computer Applications
 
PPTX
Big data analytics
Isłém Jendoubi
 
PDF
Data Analaytics.04. Data visualization
Alex Rayón Jerez
 
PDF
The Road to Data-Informed Agile Development Processes
Christoph Matthies
 
PPT
Agile softwareengineering
Mahfuz1061
 
PPTX
Agile Business Intelligence
Evan Leybourn
 
An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives
Christoph Matthies
 
Data Science towards the Digital Enterprise
Jake Bouma
 
Z group project guide
IDES 2105 Computer Applications
 
Big data analytics
Isłém Jendoubi
 
Data Analaytics.04. Data visualization
Alex Rayón Jerez
 
The Road to Data-Informed Agile Development Processes
Christoph Matthies
 
Agile softwareengineering
Mahfuz1061
 
Agile Business Intelligence
Evan Leybourn
 

Similar to Using Data to Inform Decisions in Agile Software Development (20)

PDF
Agility for big data
Charlie Cheng
 
PPTX
Increasing Analytical Thinking In Agile Teams 1.5 (1).pptx
NickFoard2
 
PDF
What is data-driven decision-making, and why is it important.pdf
Soumodeep Nanee Kundu
 
PDF
Lean Product Management @ SDEC13, Winnipeg
Dave Sharrock
 
PDF
Larry Maccherone: "Probabilistic Decision Making"
RedHatAgileDay
 
PPTX
Less Talk, More Data Driven Conversation
Gendry Morales
 
PPTX
Business Agility And Software Development Alan Chedalawada
Valtech UK
 
PDF
121211 depfac ulb_master_presentation_v5_1
Thibaut De Vylder
 
PDF
Agile methods and dw mha
AgileDenver
 
PDF
Value Driven Development by Dave Thomas
Naresh Jain
 
PPT
Agile Data Architecture
Cprime
 
PDF
Predictive Analytics & Business Insights
June Andrews
 
PDF
Ray Scott - Agile Solutions – Leading with Test Data Management - EuroSTAR 2012
TEST Huddle
 
PPTX
How to use your data science team: Becoming a data-driven organization
Yael Garten
 
PDF
Agile Data Warehousing (DW)/Business Intelligence (BI): Addressing the Hard P...
Scott W. Ambler
 
PPTX
You want it when? Probabilistic forecasting and decision making
Larry Maccherone
 
PDF
Agile teams get a grip - martijn groenewegen
Nesma
 
PDF
Estimate Value to Deliver Value: Effectively Estimate the Value of Requiremen...
Dave Sharrock
 
PDF
Thoughts on Lean Product Development at CAMUG, YYC Nov 2014
Dave Sharrock
 
PDF
DC Business Intelligentsia January Meetup: Agile BI and Data Chaos
Excella
 
Agility for big data
Charlie Cheng
 
Increasing Analytical Thinking In Agile Teams 1.5 (1).pptx
NickFoard2
 
What is data-driven decision-making, and why is it important.pdf
Soumodeep Nanee Kundu
 
Lean Product Management @ SDEC13, Winnipeg
Dave Sharrock
 
Larry Maccherone: "Probabilistic Decision Making"
RedHatAgileDay
 
Less Talk, More Data Driven Conversation
Gendry Morales
 
Business Agility And Software Development Alan Chedalawada
Valtech UK
 
121211 depfac ulb_master_presentation_v5_1
Thibaut De Vylder
 
Agile methods and dw mha
AgileDenver
 
Value Driven Development by Dave Thomas
Naresh Jain
 
Agile Data Architecture
Cprime
 
Predictive Analytics & Business Insights
June Andrews
 
Ray Scott - Agile Solutions – Leading with Test Data Management - EuroSTAR 2012
TEST Huddle
 
How to use your data science team: Becoming a data-driven organization
Yael Garten
 
Agile Data Warehousing (DW)/Business Intelligence (BI): Addressing the Hard P...
Scott W. Ambler
 
You want it when? Probabilistic forecasting and decision making
Larry Maccherone
 
Agile teams get a grip - martijn groenewegen
Nesma
 
Estimate Value to Deliver Value: Effectively Estimate the Value of Requiremen...
Dave Sharrock
 
Thoughts on Lean Product Development at CAMUG, YYC Nov 2014
Dave Sharrock
 
DC Business Intelligentsia January Meetup: Agile BI and Data Chaos
Excella
 
Ad

More from Christoph Matthies (19)

PDF
Investigating Software Engineering Artifacts in DevOps Through the Lens of Bo...
Christoph Matthies
 
PDF
Automated Exercises & Software Development Data
Christoph Matthies
 
PDF
Challenges (and Opportunities!) of a Remote Agile Software Engineering Projec...
Christoph Matthies
 
PDF
Experience vs Data: A Case for More Data-informed Retrospective Activities
Christoph Matthies
 
PDF
More than Code: Contributions in Scrum Software Engineering Teams
Christoph Matthies
 
PDF
Agile Software Development Practices: Perceptions & Project Data
Christoph Matthies
 
PDF
Counteracting Agile Retrospective Problems with Retrospective Activities
Christoph Matthies
 
PDF
Feedback in Scrum: Data-Informed Retrospectives
Christoph Matthies
 
PDF
Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching E...
Christoph Matthies
 
PDF
Scrum2Kanban: Integrating Kanban and Scrum in a University Software Engineeri...
Christoph Matthies
 
PDF
Should I Bug You? Identifying Domain Experts in Software Projects Using Code...
Christoph Matthies
 
PDF
Introduction to Lean Software & Kanban
Christoph Matthies
 
PDF
Lightweight Collection and Storage of Software Repository Data with DataRover
Christoph Matthies
 
PDF
Pybelsberg — Constraint-based Programming in Python
Christoph Matthies
 
PDF
Git Tricks — git utilities that make life git easier
Christoph Matthies
 
PDF
How to reverse engineer Android applications—using a popular word game as an ...
Christoph Matthies
 
PDF
Beat Your Mom At Solitaire—Reverse Engineering of Computer Games
Christoph Matthies
 
PDF
Introduction to Homomorphic Encryption
Christoph Matthies
 
PDF
Hacker News vs. Slashdot—Reputation Systems in Crowdsourced Technology News
Christoph Matthies
 
Investigating Software Engineering Artifacts in DevOps Through the Lens of Bo...
Christoph Matthies
 
Automated Exercises & Software Development Data
Christoph Matthies
 
Challenges (and Opportunities!) of a Remote Agile Software Engineering Projec...
Christoph Matthies
 
Experience vs Data: A Case for More Data-informed Retrospective Activities
Christoph Matthies
 
More than Code: Contributions in Scrum Software Engineering Teams
Christoph Matthies
 
Agile Software Development Practices: Perceptions & Project Data
Christoph Matthies
 
Counteracting Agile Retrospective Problems with Retrospective Activities
Christoph Matthies
 
Feedback in Scrum: Data-Informed Retrospectives
Christoph Matthies
 
Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching E...
Christoph Matthies
 
Scrum2Kanban: Integrating Kanban and Scrum in a University Software Engineeri...
Christoph Matthies
 
Should I Bug You? Identifying Domain Experts in Software Projects Using Code...
Christoph Matthies
 
Introduction to Lean Software & Kanban
Christoph Matthies
 
Lightweight Collection and Storage of Software Repository Data with DataRover
Christoph Matthies
 
Pybelsberg — Constraint-based Programming in Python
Christoph Matthies
 
Git Tricks — git utilities that make life git easier
Christoph Matthies
 
How to reverse engineer Android applications—using a popular word game as an ...
Christoph Matthies
 
Beat Your Mom At Solitaire—Reverse Engineering of Computer Games
Christoph Matthies
 
Introduction to Homomorphic Encryption
Christoph Matthies
 
Hacker News vs. Slashdot—Reputation Systems in Crowdsourced Technology News
Christoph Matthies
 
Ad

Recently uploaded (20)

PPTX
Introduction to Artificial Intelligence
lohedi9363
 
PDF
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
PPTX
ai tools demonstartion for schools and inter college
harshavardhanreddyka11
 
PPTX
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
PDF
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
PPTX
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
PDF
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
PPTX
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
PDF
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
PPTX
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
AnujKumar530915
 
PDF
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
PPTX
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
PPT
Introduction Database Management System for Course Database
ReinertYosua
 
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
Introduction to Artificial Intelligence
lohedi9363
 
PTS Company Brochure 2025 (1).pdf.......
pts464036
 
ai tools demonstartion for schools and inter college
harshavardhanreddyka11
 
Operating system designcfffgfgggggggvggggggggg
rmskumara09
 
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
Odoo Companies in India – Driving Business Transformation.pdf
azhuhardeen1968
 
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
William Chong
 
CHAPTER 2 - PM Management and IT Context
KevinPutra14
 
SAP S4 Hana Brochure 3 (PTS SYSTEMS AND SOLUTIONS)
pts464036
 
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
Audit Checklist Design Aligning with ISO, IATF, and Industry Standards — Omne...
rohnjim07
 
CHAPTER 12 - CYBER SECURITY AND FUTURE SKILLS (1) (1).pptx
AnujKumar530915
 
Wondershare Filmora 15 Crack With Activation Key [2025
ghrom2211g
 
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
Introduction Database Management System for Course Database
ReinertYosua
 
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 

Using Data to Inform Decisions in Agile Software Development

  • 1. Hasso Plattner Institute University of Potsdam, Germany christoph.matthies@hpi.de @chrisma0 Using Data to Inform Decisions in Agile Software Development Christoph Matthies, Guenter Hesse Prague, July ’19
  • 2. Motivation 2 for using data in software engineering Software engineering must shed the folkloric advice and anecdotal evidence [...] and replace them with convincing results, established with [...] empirical methods – Bertrand Meyer [Meyer, 2013] “ ”[Meyer, 2013] B. Meyer, H. Gall, M. Harman, and G. Succi, “Empirical Answers to Fundamental Software Engineering Problems (Panel),” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. New York, USA: ACM, 2013, pp. 14–18. Picture: https://commons.wikimedia.org/wiki/File:Bertrand_Meyer_recent.jpg
  • 3. Importance of Decision-Making 3 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
  • 4. Importance of Decision-Making 4 in software development projects ■ Software project success hinges on many factors □ Budget, scope, team organization, technology, architecture □ Customer engagement, requirements analysis ■ Correct project decisions critical for success [Drury, 2012] □ Overruns of budget and schedule □ Project cancellation [Taherdoost, 2018] [Drury, 2012] Drury, M., Conboy, K., and Power, K. (2012). Obstacles to decision making in Agile software development teams. Journal of Systems and Software, 85(6):1239–1254. [Taherdoost, 2018] Taherdoost, H. and Keshavarzsaleh, A. (2018). A Theoretical Review on IT Project Success/Failure Factors and Evaluating the Associated Risks. In Mathematical and Computational Methods in Electrical Engineering
  • 5. Motivation 5 for using data in Agile software development processes Scrum is founded on empirical process control theory [...] knowledge comes from experience and making decisions based on what is known. [...] – The Scrum Guide [Schwaber, 2017] [Schwaber, 2017] K. Schwaber, J. Sutherland, “The Scrum Guide - The Definitive Guide to Scrum,” 2017, [online] http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf Picture: https://www.scrum.org/resources/2017-scrum-guide-update-ken-schwaber-and-jeff-sutherland “ ”
  • 6. Decision-Making in Agile 6 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
  • 7. Decision-Making in Agile 7 Evidence-based decision-making in teams ■ Agile teams should make use of available data to enable “evidence-based decision making” [Fitzgerald, 2014] ■ However, few details available on concrete implementations Main Research Questions ■ What data is available and relevant to Agile teams? ■ What methods are transferable from other domains? [Fitzgerald, 2014] Fitzgerald, B., Musiał, M., and Stol, K.-J. (2014). Evidence-based decision making in lean software project management. In Companion Proceedings of the 36th International Conference on Software Engineering - ICSE Companion 2014, pages 93–102, New York, New York, USA. ACM Press.
  • 8. Related Work [Svensson, 2019] 8 Unfulfilled Potential of DDDM in Agile [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
  • 9. Potential of DDDM 9 Lots of potential, little usage [Svensson, 2019] ■ Survey of software practitioners ■ How is data used in the company for making decisions? [Svensson, 2019] Svensson, R.B., Feldt, R., & Torkar, R. “The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development”, 20th International Conference on Agile Software Development (XP), 2019 (preprint), https://arxiv.org/abs/1904.03948
  • 10. Data-Driven Decision-Making 10 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
  • 11. Data-Driven Decision-Making 11 In business contexts ■ DDDM contrasted with “gut feeling” [Provost, 2013] □ Standard in business in the absence of analysis technologies ■ Technology use in business has fueled DDDM adoption □ Increased company productivity [Brynjolfsson, 2011] □ Transparency & real-time information intensifies tensions, management must adapt [Schrage, 2016] [Provost, 2013] Provost, F. and Fawcett, T. (2013). Data Science and its Relationship to Big Data and Data-Driven Decision Making. Big Data, 1(1):51–59. [Brynjolfsson, 2011] Brynjolfsson, E., Hitt, L. M., and Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. [Schrage, 2016] Schrage, M. (2016). How the big data explosion has changed decision making. Harvard Business Review Digital Articles, pages 2–5
  • 12. Agile Software Development 12 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
  • 13. Agile Software Development 13 Development Processes and their connection to data ■ Plan-driven, rational decision-making [Moe, 2012] □ Sets of rules that govern behavior □ Assumption: outcome probabilities are known [Zannier, 2007] ■ Data enables teams to implement Agile feedback cycles ■ Agile, naturalistic decision-making □ Situational behavior, context-dependent rules [Moe, 2012] Moe, N. B., Aurum, A., and Dybå, T. (2012). Challenges of shared decision-making: A multiple case study of agile software development. Information and Software Technology, 54(8):853–865. [Zannier, 2007] Zannier, C. and Maurer, F. (2006). Foundations of Agile Decision Making from Agile Mentors and Developers. In Abrahamsson, P., Marchesi, M., and Succi, G., editors, Extreme Programming and Agile Processes in Software Engineering, pages 11–20, Berlin, Heidelberg. Springer Berlin Heidelberg.
  • 14. Levels of Decisions 14 The impact of Agile “self-organizing” teams ■ Strategic plans define long-term goals ■ Tactical plans involve managing projects to achieve goals ■ Operational plans define milestones, and the actual work
  • 15. Data Use in Business 15 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
  • 16. Data Use in Business 16 A wealth of data to use for decisions ■ Businesses gather detailed statistics □ Users, suppliers, partners & competitors □ ERP systems for collection, BI for analysis [Chen, 2012] ■ Continuing digitization allows additional data collection □ “Reality mining”: cars, smart devices, etc. [Svensson, 2019] □ Clickstream data, UI interactions, high time-resolution [Chen, 2012] Chen, H., Chiang, R. H. L., and Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS quarterly, 36(4). [Svensson, 2019] Svensson, R. B., Feldt, R., and Torkar, R. (2019). The Unfulfilled Potential of Data-Driven Decision Making in Agile Software Development. In Agile Processes in Software Engineering and Extreme Programming, pages 69–85. Springer International Publishing.
  • 17. Data in Software Development 17 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
  • 18. Data in Software Development 18 Much more than code ■ Software development process itself: source of valuable data □ Intrinsically relevant to making decisions □ Not only code, but a multitude of supporting artifacts ■ Processes are “inscribed” into software artifacts [de Souza, 2005] □ E.g. VCS: who, when, what, why, status □ Also includes evidence of failures [Ziftci & Reardon, 2017] [de Souza, 2005] de Souza, C., Froehlich, J., and Dourish, P. (2005). Seeking the Source: Software Source Code as a Social and Technical Artifact. In Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work - GROUP ’05, page 197, New York, New York, USA. ACM, ACM Press. [Ziftci & Reardon] Ziftci, C. and Reardon, J. (2017). Who broke the build? Automatically identifying changes that induce test failures in continuous integration at google scale. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering in Practice Track, ICSE-SEIP 2017, pages 113–122.
  • 19. Software Project Data 19 [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
  • 20. Software Project Data 20 “a gold-mine of actionable information” [Guo et al., 2016] Primary purpose: communication [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016.
  • 21. Software Project Data 21 Use this data for reflection [Guo et al., 2016] Guo, J., Rahimi, M., Cleland-Huang, J., Rasin, A., Hayes, J. H., & Vierhauser, M, “Cold-start software analytics”, In Proceedings of the 13th International Workshop on Mining Software Repositories - MSR ’16, pp. 142–153, New York, New York, USA, ACM Press, 2016. “a gold-mine of actionable information” [Guo et al., 2016]
  • 22. Scrum Process 22 Where Agile decision-making happens [Schwaber & Sutherland, 2017] 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review [Schwaber & Sutherland, 2017] Schwaber, K., & Sutherland, J, “The Scrum Guide - The Definitive Guide to Scrum: The Rules of the Game”, 2017, [online] Available: http://scrumguides.org/docs/scrumguide/v2017/2017-Scrum-Guide-US.pdf
  • 24. Sprint Planning 24 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence
  • 25. Sprint Planning 25 Building a backlog of work for the next Sprint ■ Selection and prioritization of features for next iteration □ Based on stakeholders’ experiences & opinions [Svensson, 2019] □ Lack explanation and links to evidence ■ Enable short feedback cycles between devs and real customers □ Metrics from deployed software, real use cases □ Observe actual (mis)usage, engineering relevant metrics □ Base changes on results
  • 26. Daily Scrum 26 Quick status update 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
  • 27. Daily Scrum 27 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed
  • 28. Daily Scrum 28 Quick status update ■ Short daily status meeting □ “What did I do?”, “What will I do?”, “Blockers?” □ Strictly time-boxed ■ Development artifacts hold information on what was done □ Concrete items that were completed □ Rely less on personal recollection □ Optimize for speed
  • 29. Sprint Retrospective 29 Reflecting on past work and improving 1 2 3 4 5 6 7 Sprint Planning Sprint BacklogBacklog Sprint Daily Stakeholder Input Product 4 iterations Sprint Retrospective Finished Increment Sprint Review
  • 30. Retrospectives 30 Reflecting on past work and improving What should be improved? Retrospective Meeting
  • 31. 31 Did we improve what we planned? Retrospective Meeting Retrospectives Reflecting on past work and improving
  • 32. 32 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
  • 33. 33 Did we improve what we planned? Retrospective Meeting Project DataEvidence of last iteration’s work code docs stats Retrospectives Reflecting on past work and improving
  • 34. Related Research Fields 34 Mining Software Repositories ■ Draw from MSR techniques [Prana, 2018] ■ MSR: Focus on large amounts of repositories □ “most widely used open source license?” [Dyer, 2013] ■ Little research: Few repositories, intricate knowledge of creators / users [Prana, 2018] Prana, G. A. A., Treude, C., Thung, F., Atapattu, T., & Lo, D. “Categorizing the Content of GitHub README Files”. Empirical Software Engineering. 2018. https://doi.org/10.1007/s10664-018-9660-3 [Dyer, 2013] Dyer, R., Nguyen, H. A., Rajan, H., & Nguyen, T. N. “Boa: A language and infrastructure for analyzing ultra-large-scale software repositories”. In Proceedings - International Conference on Software Engineering. pp. 422–431. 2013. IEEE.
  • 35. Conclusion 35 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
  • 36. Conclusion 36 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
  • 37. Conclusion 37 ■ Agile requires teams to make decisions based on best information available [Schuh, 2004] ■ Definition of “best information” ongoing research challenge ■ Individuals can ignore analyses or own preferences due to rules, traditions or influence of others [Dybå and Dingsøyr, 2008] Data analysis and interpretation by Agile teams (supported by context changes) can enable better informed decision [Dybå and Dingsøyr, 2008] Dybå, T. and Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833–859. [Schuh, 2004] Schuh, P. (2004). Integrating Agile Development In The Real World. Charles River Media, Inc., Rockland, MA, USA, 1st edition.
  • 39. Image Credits 39 In order of appearance ■ Data directed Decision Making by H Alberto Gongora from the Noun Project (CC BY 3.0 US) ■ Project by useiconic.com from the Noun Project (CC BY 3.0 US) ■ Question by unlimicon from the Noun Project (CC BY 3.0 US) ■ Business by David Glöckler from the Noun Project (CC BY 3.0 US) ■ Agile Development by Bê Sabino from the Noun Project (CC BY 3.0 US) ■ Digitization by priyanka from the Noun Project (CC BY 3.0 US) ■ Developer by becris from the Noun Project (CC BY 3.0 US) ■ Plan by Gregor Cresnar from the Noun Project (CC BY 3.0 US) ■ Daily Scrum by Swen-Peter Ekkebus from the Noun Project (CC BY 3.0 US) ■ Documents by LUFTFI GANI AL ACHMAD from the Noun Project (CC BY 3.0 US) ■ Idea Generation by priyanka from the Noun Project (CC BY 3.0 US)