SlideShare a Scribd company logo
Software Startup Engineering: A Systematic Mapping
Study
Anh Nguyen-Duc, Vebjørn Berg, Jørgen Birkeland,
Ilias Pappas, Letizia Jaccheri
EMSE Summer
school 2018
1
About myself
• 2011-2014, Ph.D from Norwegian University of
Science and Technology
– PhD title “Supporting coordination of software
development across organizational boundaries”
• 2015 – currently, entrepreneurs, startup
enthusiast, startup researcher
– Software Startup Labs
• 2017 – currently, Associate Professor at University
of South Eastern Norway
– Human Computer Interaction, Project driven
courses
Motivation
Case - Muml
• Vision: To be the fastest provider of validated breaking news - by
capitalizing on live, super-local user-generated content + validation
technology
• Funding:
– 100k Eur from Innovation Norway,
– 50k Usd from Google News Initiatives
– 50k Nok self-funding
• Outsourced development team in Asia
• A fully functional product
4
Case - Muml
5
Case - Muml
• Terminated operation after 2 years 7 months 18 days
• Reasons for failures
– Explainable factors
• Human factors
• Financial factors
• Process factors
• Technology factors
– Unexplainable factors
6
Certain failures can be avoid by learning
either from past experiences or systematic
and external knowledge
Startups vs. SMEs
Startup companies are unique:
Little or no operating history
Limited resources
Multiple influences
Dynamic technologies and markets
7
Motivations
• Startup researchers have called for a further attention
to engineering approaches
• Systematic mapping study helps to identify the current
status in the area and pave the way for more empirical
studies examining startups
8
N. Paternoster, C. Giardino, M. Unterkalmsteiner, T. Gorschek, P. Abra- hamsson, Software
development in startup companies: A systematic map- ping study, Information and Software Technology
56 (10) (2014)
E. Klotins, M. Unterkalmsteiner, T. Gorschek, Software Engineering Knowledge Areas in Startup
Companies: A Mapping Study, Vol. 210 of Lecture Notes in Business Information Processing, 2015, pp.
Research Questions
• RQ1: How has software startup research changed over time in
terms of focused knowledge areas?
• RQ2: What is the relative strength of the empirical evidence
reported?
• RQ3: In what context has software startup research been
conducted?
9
Research Methodology
10
Research Methodology
11
Results
RQ1: How has software startup
research changed over time in
terms of focused knowledge areas?
Focus areas are software process,
management, construction, design, and
requirements, with the shift of focus toward
process and management areas.
RQ2: What is the relative strength
of the empirical evidence reported?
The rigor of primary papers was higher
between 2013-2017 than that of 1994-2013.
RQ3: In what context has software
startup research been conducted?
Thematic concepts representing the software
startup context include innovation, lack of
resources, uncertainty, time-pressure, small
team, highly reactive, and rapidly evolving.
12
RQ1- Publication Frequency, 1994-2017 (n=74)
13
RQ1- SE Knowledge area coverage
14
RQ1- SE Process
• The need for adapting software development process to startup context:
– Contextual factors: project’s scope, magnitude, complexity, and changing
requirements
• The lack of guidance how startups can establish their methods
• More studies to contribute to the adoption of agile practices in startups
15
RQ1- SE Professional Practices
• Startup processes need to cover both business and engineering aspects
• Startup developers need to acquire not only technical competence but also
business mindset
• Developers in software startups typically prioritize speed related agile practices
rather than quality related ones
• A possible research area is to investigate how universities can facilitate learning and
to support the specific needs of practitioners that are to work in software startups
16
RQ1- SE Management
• Relates software project management to business model experimentation and
customer development
• Startups struggle with how pivoting should be performed at diferent lifecycle
stages?
• Startup-aware outsourcing is a feasible option for early-stage startups
• A need for more research to identify how startups explicitly manage risks
17
RQ1- Software Requirements
• Requirements mainly were elicited through the founders’ assumptions and
interpretations of the market
• Minimum Viable Products (MVPs) are effective tools for requirements elicitation
• The lack of studies investigating how software startups perform requirements
engineering processes
18
RQ1- Software Design
• Requirements mainly were elicited through the founders’ assumptions and
interpretations of the market
• The lack of studies investigating how software startups perform requirements
engineering processes
19
RQ1- Software Quality
• Testing is critical to startups’ success, but often overlooked
• The most common testing techniques:
– unit tests (37%)
– pilot clients (25%)
– functional tests (25%)
– specialist testers (13%)
• Future research on how startups can learn from established companies’ systematic
testing processes
20
RQ1- Software Construction
• There does not exist a clear understanding of how entrepreneurs can use the
different tools efficiently to meet their specific needs
• A software tool portal can be helpful to support software construction
21
RQ2- Type of research publications
22
RQ2- Rigour of papers in 2013-2017 (n=27)
23
RQ2- Rigour of papers in 1994-2013 (n=47)
24
RQ3- Thematic Concepts, 1994-2017
Thematic Concepts
Frequency 13’-
17’ (#27)
Frequency 94’-
13’ (#47)
Innovation/Innovative 15 19
Uncertainty 14 15
Small team 11 12
Lack of resources 9 21
Little working/operating history 9 3
Time-pressure 7 17
Rapidly evolving 5 16
New company 5 8
Highly reactive 3 19
Highly risky 3 8
Third party dependency 2 12
One product 2 9
Not self-sustained 1 3
Low-experienced team 0 9
Flat organisation 0 5
25
Final remarks
• Software startups find it hard to apply theory in practice
• Future work on certain research themes, i.e startup evolution models,
human aspects, and consolidation of contextual factors
• Multivocal Literature Review can be the next review on software startup
26

More Related Content

PPTX
Software Project Management
PPTX
Introduction project managemen
PPT
Project Gestalt Program Success
PDF
Process trends
PDF
Agile Software Development Practices: Perceptions & Project Data
PPTX
Scope management
Software Project Management
Introduction project managemen
Project Gestalt Program Success
Process trends
Agile Software Development Practices: Perceptions & Project Data
Scope management

What's hot (20)

PDF
LeroWorkshopJaanaNevalainen
PPTX
Software Project Management
PPT
5 The Logical Framework - a short course for NGOs
PPT
Proposal writing resource the logframe approach
PPTX
It project risk management
PPT
Logical frameworks
PPT
Project cycle
PDF
Project breathalyzer
PPT
Logical framework
PPTX
Evaluability Assessments and Choice of Evaluation Methods
PPT
Software engg. pressman_ch-21
DOC
Logical framework guide
DOCX
Software project planning and monitoring
PPT
Project management concepts
PPT
Gurjyot
PPTX
Project management
PDF
RTF
ManishaResume latest
PPT
Software Project Management
PPT
Overview of project planning
LeroWorkshopJaanaNevalainen
Software Project Management
5 The Logical Framework - a short course for NGOs
Proposal writing resource the logframe approach
It project risk management
Logical frameworks
Project cycle
Project breathalyzer
Logical framework
Evaluability Assessments and Choice of Evaluation Methods
Software engg. pressman_ch-21
Logical framework guide
Software project planning and monitoring
Project management concepts
Gurjyot
Project management
ManishaResume latest
Software Project Management
Overview of project planning
Ad

Similar to Software Startup Engineering: A Systematic Mapping Study (20)

PDF
Software Analytics
PPT
PPTX
Introduction to Software Engineering.pptx
PPTX
Software Engineering CST357 Lecture 1.pptx
PPTX
Co-op Presentation Fall_Winter_2014 FINAL VERSION
PPTX
Project Management - Foundation
PDF
KEY DETERMINANTS FOR SUCCESSFUL PROJECT IMPLEMENTATION IN THE INDUSTRIAL AREA...
PPTX
DCSUG - Finding Lean in Agile
PDF
Finding Lean in Agile by Adam Parker
PPTX
Unit 1 spm
PPTX
Unit 1 spm
PPTX
1_slides-bài-giảng-SoftwareProjectManagement.pptx
PPTX
intro.pptx
PDF
How to Get to ‘One Source of Truth’ on Large, Multi-Year Programs
PPTX
1_slides bài giảng - SoftwareProjectManagement.pptx
PDF
Project Management
PDF
software-process-project-managementr17a0539.pdf
PDF
Software Engineering Practice - Advanced Development Methodologies
Software Analytics
Introduction to Software Engineering.pptx
Software Engineering CST357 Lecture 1.pptx
Co-op Presentation Fall_Winter_2014 FINAL VERSION
Project Management - Foundation
KEY DETERMINANTS FOR SUCCESSFUL PROJECT IMPLEMENTATION IN THE INDUSTRIAL AREA...
DCSUG - Finding Lean in Agile
Finding Lean in Agile by Adam Parker
Unit 1 spm
Unit 1 spm
1_slides-bài-giảng-SoftwareProjectManagement.pptx
intro.pptx
How to Get to ‘One Source of Truth’ on Large, Multi-Year Programs
1_slides bài giảng - SoftwareProjectManagement.pptx
Project Management
software-process-project-managementr17a0539.pdf
Software Engineering Practice - Advanced Development Methodologies
Ad

More from Anh Nguyen Duc (12)

PPTX
A preliminary study of agility in business and production – Cases of early-s...
PPTX
Achieving product market fit in startup context - The-state-of-practices and ...
PPTX
Introduction to Global Software Engineering TDT4140
PPTX
Sharing economy and Vietnam startup prospect (Kinh tế chia sẻ và cơ hội khởi ...
PPTX
Application of economic model in software maintenance
PPTX
Supporting team coordination of software development across multiple companies
PDF
On the role of boundary spanners as a team coordination mechanism in organisa...
PPT
Coordination of software development teams across organizational boundary – A...
PPTX
Forking and coordination in multi-platform development
PPT
Impact of design complexity on software quality - A systematic review
PPTX
Dispersion, coordination and performance in GSD: a systematic review
PPTX
Supporting team coordination across organizational boundary in GSD
A preliminary study of agility in business and production – Cases of early-s...
Achieving product market fit in startup context - The-state-of-practices and ...
Introduction to Global Software Engineering TDT4140
Sharing economy and Vietnam startup prospect (Kinh tế chia sẻ và cơ hội khởi ...
Application of economic model in software maintenance
Supporting team coordination of software development across multiple companies
On the role of boundary spanners as a team coordination mechanism in organisa...
Coordination of software development teams across organizational boundary – A...
Forking and coordination in multi-platform development
Impact of design complexity on software quality - A systematic review
Dispersion, coordination and performance in GSD: a systematic review
Supporting team coordination across organizational boundary in GSD

Recently uploaded (20)

PPTX
ELS-07 Lifeskills ToT PPt-Adama (ABE).pptx
PDF
Why Has Vertical Farming Recently Become More Economical.pdf
PPTX
The Evolution of Search- Behaviour.pptx
PDF
initiate-entrepreneurship-in-healthcare-service-management-in-sierra-leone.pdf
PDF
Driving Innovation & Growth, Scalable Startup IT Services That Deliver Result...
PDF
Chapter 3 - Business environment - Final.pdf
PPTX
TimeBee vs. Toggl: Which Time Tracking Tool is Best for You?
PPTX
_From Idea to Revenue How First-Time Founders Are Monetizing Faster in 2025.pptx
PDF
AI Cloud Sprawl Is Real—Here’s How CXOs Can Regain Control Before It Costs Mi...
PPTX
ENTREPRENEURSHIP..PPT.pptx..1234567891011
PDF
Chapter 1 - Introduction to management.pdf
PPTX
Peerless Plumbing Company-Fort Worth.pptx
PPT
Chap8. Product & Service Strategy and branding
PDF
Pollitrace pitch deck- Ai powered multiple species
PPTX
Process-and-Ethics-in-Research-1.potatoi
PDF
Budora Case Study: Building Trust in Canada’s Online Cannabis Market
PPTX
Daily stand up meeting on the various business
PDF
Meme Coin Empire- Launch, Scale & Earn $500K-Month_3.pdf
PDF
Investment Risk Assessment Brief: Zacharia Ali and Associated Entities
PDF
Business Risk Assessment and Due Diligence Report: Zacharia Ali and Associate...
ELS-07 Lifeskills ToT PPt-Adama (ABE).pptx
Why Has Vertical Farming Recently Become More Economical.pdf
The Evolution of Search- Behaviour.pptx
initiate-entrepreneurship-in-healthcare-service-management-in-sierra-leone.pdf
Driving Innovation & Growth, Scalable Startup IT Services That Deliver Result...
Chapter 3 - Business environment - Final.pdf
TimeBee vs. Toggl: Which Time Tracking Tool is Best for You?
_From Idea to Revenue How First-Time Founders Are Monetizing Faster in 2025.pptx
AI Cloud Sprawl Is Real—Here’s How CXOs Can Regain Control Before It Costs Mi...
ENTREPRENEURSHIP..PPT.pptx..1234567891011
Chapter 1 - Introduction to management.pdf
Peerless Plumbing Company-Fort Worth.pptx
Chap8. Product & Service Strategy and branding
Pollitrace pitch deck- Ai powered multiple species
Process-and-Ethics-in-Research-1.potatoi
Budora Case Study: Building Trust in Canada’s Online Cannabis Market
Daily stand up meeting on the various business
Meme Coin Empire- Launch, Scale & Earn $500K-Month_3.pdf
Investment Risk Assessment Brief: Zacharia Ali and Associated Entities
Business Risk Assessment and Due Diligence Report: Zacharia Ali and Associate...

Software Startup Engineering: A Systematic Mapping Study

  • 1. Software Startup Engineering: A Systematic Mapping Study Anh Nguyen-Duc, Vebjørn Berg, Jørgen Birkeland, Ilias Pappas, Letizia Jaccheri EMSE Summer school 2018 1
  • 2. About myself • 2011-2014, Ph.D from Norwegian University of Science and Technology – PhD title “Supporting coordination of software development across organizational boundaries” • 2015 – currently, entrepreneurs, startup enthusiast, startup researcher – Software Startup Labs • 2017 – currently, Associate Professor at University of South Eastern Norway – Human Computer Interaction, Project driven courses
  • 4. Case - Muml • Vision: To be the fastest provider of validated breaking news - by capitalizing on live, super-local user-generated content + validation technology • Funding: – 100k Eur from Innovation Norway, – 50k Usd from Google News Initiatives – 50k Nok self-funding • Outsourced development team in Asia • A fully functional product 4
  • 6. Case - Muml • Terminated operation after 2 years 7 months 18 days • Reasons for failures – Explainable factors • Human factors • Financial factors • Process factors • Technology factors – Unexplainable factors 6 Certain failures can be avoid by learning either from past experiences or systematic and external knowledge
  • 7. Startups vs. SMEs Startup companies are unique: Little or no operating history Limited resources Multiple influences Dynamic technologies and markets 7
  • 8. Motivations • Startup researchers have called for a further attention to engineering approaches • Systematic mapping study helps to identify the current status in the area and pave the way for more empirical studies examining startups 8 N. Paternoster, C. Giardino, M. Unterkalmsteiner, T. Gorschek, P. Abra- hamsson, Software development in startup companies: A systematic map- ping study, Information and Software Technology 56 (10) (2014) E. Klotins, M. Unterkalmsteiner, T. Gorschek, Software Engineering Knowledge Areas in Startup Companies: A Mapping Study, Vol. 210 of Lecture Notes in Business Information Processing, 2015, pp.
  • 9. Research Questions • RQ1: How has software startup research changed over time in terms of focused knowledge areas? • RQ2: What is the relative strength of the empirical evidence reported? • RQ3: In what context has software startup research been conducted? 9
  • 12. Results RQ1: How has software startup research changed over time in terms of focused knowledge areas? Focus areas are software process, management, construction, design, and requirements, with the shift of focus toward process and management areas. RQ2: What is the relative strength of the empirical evidence reported? The rigor of primary papers was higher between 2013-2017 than that of 1994-2013. RQ3: In what context has software startup research been conducted? Thematic concepts representing the software startup context include innovation, lack of resources, uncertainty, time-pressure, small team, highly reactive, and rapidly evolving. 12
  • 13. RQ1- Publication Frequency, 1994-2017 (n=74) 13
  • 14. RQ1- SE Knowledge area coverage 14
  • 15. RQ1- SE Process • The need for adapting software development process to startup context: – Contextual factors: project’s scope, magnitude, complexity, and changing requirements • The lack of guidance how startups can establish their methods • More studies to contribute to the adoption of agile practices in startups 15
  • 16. RQ1- SE Professional Practices • Startup processes need to cover both business and engineering aspects • Startup developers need to acquire not only technical competence but also business mindset • Developers in software startups typically prioritize speed related agile practices rather than quality related ones • A possible research area is to investigate how universities can facilitate learning and to support the specific needs of practitioners that are to work in software startups 16
  • 17. RQ1- SE Management • Relates software project management to business model experimentation and customer development • Startups struggle with how pivoting should be performed at diferent lifecycle stages? • Startup-aware outsourcing is a feasible option for early-stage startups • A need for more research to identify how startups explicitly manage risks 17
  • 18. RQ1- Software Requirements • Requirements mainly were elicited through the founders’ assumptions and interpretations of the market • Minimum Viable Products (MVPs) are effective tools for requirements elicitation • The lack of studies investigating how software startups perform requirements engineering processes 18
  • 19. RQ1- Software Design • Requirements mainly were elicited through the founders’ assumptions and interpretations of the market • The lack of studies investigating how software startups perform requirements engineering processes 19
  • 20. RQ1- Software Quality • Testing is critical to startups’ success, but often overlooked • The most common testing techniques: – unit tests (37%) – pilot clients (25%) – functional tests (25%) – specialist testers (13%) • Future research on how startups can learn from established companies’ systematic testing processes 20
  • 21. RQ1- Software Construction • There does not exist a clear understanding of how entrepreneurs can use the different tools efficiently to meet their specific needs • A software tool portal can be helpful to support software construction 21
  • 22. RQ2- Type of research publications 22
  • 23. RQ2- Rigour of papers in 2013-2017 (n=27) 23
  • 24. RQ2- Rigour of papers in 1994-2013 (n=47) 24
  • 25. RQ3- Thematic Concepts, 1994-2017 Thematic Concepts Frequency 13’- 17’ (#27) Frequency 94’- 13’ (#47) Innovation/Innovative 15 19 Uncertainty 14 15 Small team 11 12 Lack of resources 9 21 Little working/operating history 9 3 Time-pressure 7 17 Rapidly evolving 5 16 New company 5 8 Highly reactive 3 19 Highly risky 3 8 Third party dependency 2 12 One product 2 9 Not self-sustained 1 3 Low-experienced team 0 9 Flat organisation 0 5 25
  • 26. Final remarks • Software startups find it hard to apply theory in practice • Future work on certain research themes, i.e startup evolution models, human aspects, and consolidation of contextual factors • Multivocal Literature Review can be the next review on software startup 26

Editor's Notes

  • #4: 60% of startups do not survive in the first five years 75% of venture capital funded startups fail
  • #8: startup as a temporary organization that seeks a scalable, repeatable, and profitable business model, and therefore aims to grow
  • #10: Lean manufacturing in toyota Lean prototyping
  • #13: Researchers have provided lessons learned and advice studies, paying less attention to specific tools and frameworks. Two reasons for this are increased importance of startups, and increased focus on researchers providing high-quality research. Startup literature provides an inconsistent use of thematic concepts describing startups.
  • #14: From 1994-2013, the highest number of primary papers within a single year was 7 (2008). In comparison, 2016 and 2017 constituted 9 and 11 papers respectively.
  • #15: Between 1994-2013 “software design” and “software requirements” are the most represented knowledge areas, However, software engineering process” and “software management” have received significant attention from the community between 2013-2017.
  • #16: Lean manufacturing in toyota Lean prototyping
  • #17: Lean manufacturing in toyota Lean prototyping
  • #18: Lean manufacturing in toyota Lean prototyping
  • #19: Lean manufacturing in toyota Lean prototyping
  • #20: Lean manufacturing in toyota Lean prototyping
  • #21: Lean manufacturing in toyota Lean prototyping
  • #22: Lean manufacturing in toyota Lean prototyping
  • #23: Figure 7 shows the contribution types of primary papers between 1994-2017, separating the periods before and after 2013. The most frequently provided contribution types between 1994-2013 were advice and model, while lessons learned was most represented between 2013-2017. The least frequently used ones combined from both studies were framework, guidelines, and tools.
  • #26: Startup Start-up Very small entity Very small company Very small enterprise It is usual that researchers specify the product orientation of the startups (e.g., B2B/B2C). The number of startups under investigation is in the range from 1-20 startups. The most frequently used number of startups was found to be 3-5. The number of employees is usually in the range of 2-25, depending on the lifecycle stage of the company. The age of the investigated companies is usually in the range 1 month to 3 years. Startups use different software development methods. No more than two papers mentioned whether the investigated companies had received any funding