REfine a gamifiedplatform for participatory requirements engineering

REfine: A Gamified Platform for
Participatory Requirements Engineering
R. Snijders, F. Dalpiaz, S. Brinkkemper (Utrecht University, NL)
Mahmood Hosseini, Raian Ali (Bournemouth University, UK)
Atilla Özüm (KPMG Technology Advisory, NL)

Agenda
1. Stakeholders involvement in RE
2. The potential: crowdsourcing, gamification
3. The REfine platform
4. Results from a case study
5. Conclusions and FutureWork
REfine: a gamified platform for participatory RE

 Software projects keep failing, or being challenged
 Results from the 2014 Chaos Standish Report
16.2
31.1
52.7
Successful Cancelled Challenged

 What are the key determinants for success/failure?
Taken from the 2014
Standish Chaos Report

 But what can stakeholder/user involvement deliver?
 More accurate requirements
 Expensive and unnecessary features avoided
 Improved acceptance (Kujala, 2003)
 Higher quality of requirements
 Higher chances of project success (Kujala et al., 2005)
 Improved loyalty
 Broadened market (Kabbedijk et al., 2009)

2.The potential:
crowdsourcing, gamification

 Involving the crowd! Relying on a crowd of humans to
get things done!
 Feature requests (Win10)
 Crowdwork (Mturk)

 Gamify it!The trend is manifest
 4Square, Stackoverflow, LinkedIn, etc.
 Gamification:“the use of game design elements in non-
game contexts” (Deterding et al., 2011)
 Intrinsic kinds of motivation
 Personal gratification
 Social status

3.The REfine platform

 An interactive platform to elicit and refine requirements
 Crowd-centric: all stakeholders are welcome to participate
 Gamified: incentives through game elements

 Discussing needs via REfine (perform actions)
 Agree/disagree with needs
 Add comments
 “Liking” comments
 Branching needs
 Actions have a cost
 Money obtained
when others find
one’s contributions
useful

 Leaderboards to determine the most useful participants
 General leaderboard (overall points)
 Per type of contribution: ideator, assessor, commenter
 Per need

 The CCRE method
 From feasibility to focus group
 Fits in agile development (sprints)
 Highly iterative, despite figure
 Main characteristics
 Determines if crowd involvement
has a potential
 Uses an interactive platform like REfine
 Rewards the most active stakeholders
by inviting them to the focus groups

 The Qubus software product
 A governance, risk, and compliance tool developed by KPMG
 Under the hood, it hosts a workflow management system
 Object of our study: needs for version 7 beta

 Crowdsourcing preparation
 Three-steps invitation: internal stakeholders, clients & users,
off-stage actors
 Small crowd involved (19 people)
 Crowd involvement
 21 needs, 37 comments, 130 votes
 Little moderation
 One-month period

Stakeholder type N Active Needs Comments Votes Points
Avg. Total Avg. Total Avg. Total Avg. Total
Community
manager
1 - 2 2 8 8 21 21 41 41
Product
management
2 67% 0 0 3 6 6 12 13.5 27
Development team 4 100% 0.8 3 3.8 15 14.8 59 26.8 107
Experts 4 100% 1 4 1.3 5 2.5 10 16 64
Client 1 100% 0 0 0 0 0 0 3 3
Users 1 6% 1 1 0 0 3 3 11 11
Off-stage actors 6 86% 1.8 11 0.5 3 4.2 25 20 120
Total 19 50% 1.1 21 1.9 37 6.8 130 19.6 373

 Questionnaire: comparing with previous experiences in
communicating requirements
0% 20% 40% 60% 80% 100%
Difficulty
Usefulness
Engagement
1 2 3 4 5
3.88 (more engaging)
4.19 (more useful)
2.69 (as difficult)

 Questionnaire: perceived experience/usefulness of Refine
3.47
2.82
4.12
1.92
4.71
4.41
2.94

 Evaluation with experts from the industry (generality)
 1 = strong disagreement
 7 = strong agreement

5. Conclusions and FutureWork

5. Conclusions and Future Work
 User involvement is crucial for project success
 Potential: crowdsourcing and gamification
 The REfine tool
 Gamification to maximize stakeholder involvement
 Mixed results from a case study
 Gamification improves the experience
 It is difficult to attract a large crowd

5. Conclusions and Future Work
 Large-scale studies with more stakeholders
 Long-term incentives (real-life gamification)
 How to assess the actual effect of gamification?
 Ongoing work using user stories

Thanks for listening!
Questions?
mhosseini@bournemouth.ac.uk
f.dalpiaz@uu.nl

References
 S. Deterding, D. Dixon, R. Khaled, and L. Nacke,“From Game Design Elements to
Gamefulness: Defining Gamification,” in Proc. of MindTrek: Envisioning Future Media
Environments, 2011, pp. 9–15
 J. Kabbedijk, S. Brinkkemper, S. Jansen, and B. van derVeldt,“Customer Involvement
in Requirements Management: Lessons from Mass Market Software Development,”
in Proc. of RE,Aug. 2009, pp. 281–286
 S. Kujala,“User Involvement:A Review of the Benefits and Challenges,” Behaviour &
InformationTechnology,2003
 S. Kujala, M. Kauppinen, L. Lehtola, andT. Kojo,“The Role of User Involvement in
Requirements Quality and Project Success,” in Proc. of RE, 2005, pp. 75–84
 R. Snijders, F. Dalpiaz, M. Hosseini,A. Shahri, and R.Ali,“Crowd-Centric
Requirements Engineering,” in Proc. of the 2nd InternationalWorkshop on
Crowdsourcing and Gamification in the Cloud,2014

REfine a gamifiedplatform for participatory requirements engineering

More Related Content

Similar to REfine a gamifiedplatform for participatory requirements engineering (20)

More from Engineering and Social Informatics (ESOTICS) (20)

Recently uploaded (20)

REfine a gamifiedplatform for participatory requirements engineering