SlideShare a Scribd company logo

Survey on Requirements Engineering Tools

J
jnicolasros

This document describes a survey conducted on requirements engineering (RE) tools. It provides background on related surveys and establishes a framework of RE tool capabilities based on ISO standards. The researchers aimed to investigate the state-of-the-art of RE tools using a 146-item questionnaire sent to 94 tool representatives. The questionnaire addressed the tools' support for various RE activities based on the framework. The responses would provide an updated overview of RE tools' capabilities.

BusinessTechnology
1 of 53
Downloaded 171 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
Survey on Requirements Engineering Tools 1 / 26 Survey on Requirements Engineering Tools REFSQ Industry Track 2011 Juan M. Carrillo de Gea Christof Ebert jmcdg1@um.es, Vector, Stuttgart, Germany Joaquín Nicolás, José L. Fernández Alemán, Aurora Vizcaíno Ambrosio Toval Universidad de Castilla-La Universidad de Murcia, Spain Mancha, Spain Essen, Germany, March 29, 2011
Survey on Requirements Engineering Tools 2 / 26 Contents 1 Introduction 2 Related work 3 Classification framework for RE tools’ capabilities 4 Research methodology Research goals Instrumentation Experimental procedure 5 Results Participants Experimental results Discussion Threats to validity 6 Conclusions and future work Conclusions Future work
Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
Survey on Requirements Engineering Tools 4 / 26 Related work Related work Surveys on RE tools Surveys on RE [Zowghi and Coulin, 2005] [Liu et al., 2010] [Alenljung and Persson, 2008] [Winkler and von Pilgrim, 2010] [Hall, 2008] [Carlshamre et al., 2001] [Gregoriades and Sutcliffe, 2005] [Johansson et al., 2001] [Maiden et al., 2006] [Benslimane et al., 2007] [Portillo Rodríguez et al., 2010]
Survey on Requirements Engineering Tools 5 / 26 Classification framework for RE tools’ capabilities Regulations and guidelines ISO/IEC TR 24766:2009 Technical Report (TR) of Type 2 Set of capabilities that the RE tools should support Six major categories Requirements elicitation Requirements analysis Requirements specification Requirements V&V Requirements management Other tool capabilities Supplements ISO/IEC 14102:2008 International Standard Evaluation of CASE tools
Survey on Requirements Engineering Tools 6 / 26 Classification framework for RE tools’ capabilities RE tools’ capabilities ISO/IEC TR 24766 tool capabilities Category Amount Requirements elicitation 37 Requirements analysis 36 Requirements specification 16 Requirements verification and validation 34 Requirements management 17 Other tool capabilities 17 Total 157
Survey on Requirements Engineering Tools 7 / 26 Research methodology Research goals Goal/Question Metric (GQM) framework Goal/Question Metric (GQM) framework [Basili and Rombach, 1988] GQM template [Basili et al., 1999] Goal: To investigate the state-of-the-art on RE tools using a questionnaire aimed at software vendors Question: Do current RE tools address industry challenges? Metric: Capabilities of RE tools (questionnaire scores)
Survey on Requirements Engineering Tools 8 / 26 Research methodology Instrumentation Instrumentation I Databases hosting RE tools lists (July, 2010–August, 2010) Database Amount Ian Alexander 67 Alarcos Research Group 7 INCOSE 34 Ludwig Consulting Services 40 Qaguild 7 Volere 71 @WEBO 41 Total sample size after discarding invalid tools 94
Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
Survey on Requirements Engineering Tools 10 / 26 Research methodology Experimental procedure Experimental procedure Questionnaire and survey system preparation: September, 2010–November, 2010 Tool representatives filled in the survey: December 20, 2010–January 15, 2011 Follow-up email and deadline extension to February 7, 2011
Survey on Requirements Engineering Tools 11 / 26 Results Participants Participants I 38 participants out of 94 candidates invited (40.42%) Acclaro DFSS Leap SE CASE Spec Aligned Elements MacA&D/WinA&D Cognition Cockpit Avenqo PEP MKS Integrity Cradle Blueprint PACE G-MARC Bright Green Projects Polarion inteGREAT Caliber RM Requirements IRQA Cameo Psoda jUCMNav Requirements+ QFDcapture
Survey on Requirements Engineering Tools 12 / 26 Results Participants Participants II QPack RTIME TestTrack RM RaQuest RequisitePro TopTeam Analyst Rational DOORS RMTrak TraceCloud ReqMan Rommana TrackStudio Reqtify Scenario Plus VisibleThread On- Requirements premise/On-demand SpiraTeam Composer
Survey on Requirements Engineering Tools 13 / 26 Results Experimental results Administrative information I Year of first release Year of last release 8 30 7 25 6 5 20 4 15 3 10 2 1 5 0 0 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2008 2009 2010 2011 No. of tools No. of tools
Survey on Requirements Engineering Tools 14 / 26 Results Experimental results Administrative information II Platform required Type of license Windows/ Windows/ Windows/ Open- Open- Mac OS Linux Mac source and source and OS/Linux free not-free Windows/ Proprietary Mac OS/ and free UNIX/Linux Windows/ UNIX/Linux Windows Web-based Proprietary and not- free
Survey on Requirements Engineering Tools 15 / 26 Results Experimental results Administrative information III Cost per individual license Amount of licenses in use 18 14 16 12 14 10 12 10 8 8 6 6 4 4 2 2 0 0 No answer Less than 100 to 500 501 to More than No answer 0 to 100 101 to 1001 to More than 100 1000 1000 1000 10000 10000 Cost No. of licenses
Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
Survey on Requirements Engineering Tools 17 / 26 Results Experimental results Technical information I Tools’ scores 20 18 18 17 17 16 16 14 14 1212 12 12 12 11 11 10 10 10 9 8 8 7 6 6 6 6 5 5 5 5 4 4 4 4 4 3 3 3 2 2 2 1 1 1 1 1 0 0 0 0 Very high High Medium Low Very low Elicitation Analysis Specification Modelling V&V Management Traceability Other tool capabilities
Survey on Requirements Engineering Tools 18 / 26 Results Experimental results Technical information II Global score (level of accomplishment of the entire ISO/IEC TR 24766)—only calculated for those tools participating in all categories 12 10 8 6 4 2 0 Very high High Medium Low Very low Global
Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
Survey on Requirements Engineering Tools 21 / 26 Results Discussion Scenario 3: Requirements traceability Generating reports that compare current and previous versions when a source document is updated (55%) Tracing across the tools’ boundaries (55%) Tracing text to graphics (52%), graphics to graphics (47%), elements within graphics (42%), tables and cells within a table (36%) Generating reports of traceability attributes (60%) Best: CASE Spec, Cockpit, Cradle, G-MARC, inteGREAT, Reqtify Close to the best: Avenqo PEP, IRQA, Polarion Requirements, QPack, TopTeam Analyst
Survey on Requirements Engineering Tools 21 / 26 Results Discussion Scenario 3: Requirements traceability Generating reports that compare current and previous versions when a source document is updated (55%) Tracing across the tools’ boundaries (55%) Tracing text to graphics (52%), graphics to graphics (47%), elements within graphics (42%), tables and cells within a table (36%) Generating reports of traceability attributes (60%) Best: CASE Spec, Cockpit, Cradle, G-MARC, inteGREAT, Reqtify Close to the best: Avenqo PEP, IRQA, Polarion Requirements, QPack, TopTeam Analyst
Survey on Requirements Engineering Tools 22 / 26 Results Discussion Scenario 4: Other features Providing Eclipse support (42%) Providing data federation (28%) Providing an open data model (18%)
Survey on Requirements Engineering Tools 23 / 26 Results Threats to validity Threats to validity Internal validity (causal relationships) Mortality percentage (7.32%) Commitment of the RE tools representatives Truthfulness of the answers (Hawthorne effect) Triangulation technique Questionnaire planning and design External validity (generalizations) Participants are representative of the RE tools’ community
Survey on Requirements Engineering Tools 23 / 26 Results Threats to validity Threats to validity Internal validity (causal relationships) Mortality percentage (7.32%) Commitment of the RE tools representatives Truthfulness of the answers (Hawthorne effect) Triangulation technique Questionnaire planning and design External validity (generalizations) Participants are representative of the RE tools’ community
Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
Survey on Requirements Engineering Tools 26 / 26 Thank you for your attention Thank you! Any questions?
Survey on Requirements Engineering Tools 26 / 26 Thank you for your attention Thank you! Any questions?
Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
Survey on Requirements Engineering Tools 28 / 26 Bibliography Bibliography I Alenljung, B. and Persson, A. (2008). DESCRY: a method for evaluating decision-supporting capabilities of requirements engineering tools. In Proc. of the 14th Int. Conf. on Requir. Eng.: Foundation for Software Quality, pages 52–57. Springer-Verlag. Basili, V. and Rombach, H. (1988). The TAME project: towards improvement-oriented software environments. IEEE Trans. Softw. Eng., 14:758–773. Basili, V. R., Shull, F., and Lanubile, F. (1999). Building knowledge through families of experiments. IEEE Trans. Softw. Eng., 25:456–473.
Survey on Requirements Engineering Tools 29 / 26 Bibliography Bibliography II Benslimane, Y., Cysneiros, L. M., and Bahli, B. (2007). Assessing critical functional and non-functional requirements for web-based procurement systems: a comprehensive survey. Requir. Eng., 12:191–198. Carlshamre, P., Sandahl, K., Lindvall, M., Regnell, B., and Dag, J. N. (2001). An industrial survey of requirements interdependencies in software product release planning. In Proc. of the Fifth IEEE Int. Symp. on Requir. Eng., pages 84–92, Washington, DC, USA. IEEE Computer Society. Gregoriades, A. and Sutcliffe, A. (2005). Scenario-based assessment of nonfunctional requirements. IEEE Trans. Softw. Eng., 31:392–409.
Survey on Requirements Engineering Tools 30 / 26 Bibliography Bibliography III Hall, R. J. (2008). A method and tools for large scale scenarios. Autom. Softw. Eng., 15:113–148. Johansson, E., Wesslén, A., Bratthall, L., and Höst, M. (2001). The importance of quality requirements in software platform development - a survey. In Proc. of the 34th Annual Hawaii Int. Conf. on Sys. Sci., volume 9, Washington, DC, USA. IEEE Computer Society. Kitchenham, B. (1996). DESMET: a method for evaluating software engineering methods and tools. Tech. Report TR96-09, Dept. of Computer Science, University of Keele.
Survey on Requirements Engineering Tools 31 / 26 Bibliography Bibliography IV Liu, L., Li, T., and Peng, F. (2010). Why requirements engineering fails: a survey report from China. In Proc. of the 2010 18th IEEE Int. Requir. Eng. Conf., pages 317–322. IEEE Computer Society. Maiden, N., Seyff, N., Grunbacher, P., Otojare, O., and Mitteregger, K. (2006). Making mobile requirements engineering tools usable and useful. In Proc. of the 14th IEEE Int. Requir. Eng. Conf., pages 26–35. Portillo Rodríguez, J., Ebert, C., and Vizcaíno, A. (2010). Technologies and tools for distributed teams. IEEE Softw., 27:10–14. Winkler, S. and von Pilgrim, J. (2010). A survey of traceability in requirements engineering and model-driven development. Softw. and Sys. Modeling, 9:529–565.
Survey on Requirements Engineering Tools 32 / 26 Bibliography Bibliography V Zowghi, D. and Coulin, C. (2005). Requirements elicitation: a survey of techniques, approaches, and tools. In Aurum, A. and Wohlin, C., editors, Engineering and Managing Software Requirements, pages 19–46. Springer.

More Related Content

PPTX
Sdpl1
sraviinthiran
 
PPT
Software Requirements Engineering Methodologies
Kiran Munir
 
PDF
Unit 2 analysis and software requirements
Azhar Shaik
 
PPTX
Software Engineering unit 2
Abhimanyu Mishra
 
DOC
Session3
Jani Kusuma
 
PPT
Software Requirements in Software Engineering SE5
koolkampus
 
PPTX
03 requirement engineering_process
University of Computer Science and Technology
 
PPTX
Sda 4
AmberMughal5
 
Sdpl1
sraviinthiran
 
Software Requirements Engineering Methodologies
Kiran Munir
 
Unit 2 analysis and software requirements
Azhar Shaik
 
Software Engineering unit 2
Abhimanyu Mishra
 
Session3
Jani Kusuma
 
Software Requirements in Software Engineering SE5
koolkampus
 
03 requirement engineering_process
University of Computer Science and Technology
 
Sda 4
AmberMughal5
 

What's hot (20)

PPTX
Slcm sharbani bhattacharya
Sharbani Bhattacharya
 
PPT
Testing and Rolling Out Enterprise Applications
Gem WeBlog
 
PDF
4 sdlc and stlc
Chandra Maddigapu
 
PPTX
Software requirement enginering
Wajid Ali
 
DOC
Good Practices For Developing User Requirements
nkaur
 
PDF
Requirement analysis with use case
Rapeepan Thawornwanchai
 
PDF
Software Requirements and Specifications
vustudent1
 
PDF
System requirements engineering
Animesh Chaturvedi
 
PPS
testing
nazeer pasha
 
PPT
Sdlc
nazeer pasha
 
PPTX
Software Engineering concept
Atamjitsingh92
 
PDF
Writing Effective Use Cases
Harsh Jegadeesan
 
PDF
Modern Elicitation Process
Rajon
 
DOCX
Sdlc process document
Pesara Swamy
 
PPTX
Requirements Engineering Processes
Ra'Fat Al-Msie'deen
 
PPTX
Software development life cycle by chitta
Chittaranjan Das
 
PDF
Sdlc tutorial
Suresh Mca
 
DOCX
Software Requirements (3rd Edition) summary
Ahmed Kamel Taha
 
PDF
Object oriented-systems-development-life-cycle ppt
Kunal Kishor Nirala
 
PPTX
software engineering
Snow Queenzz
 
Slcm sharbani bhattacharya
Sharbani Bhattacharya
 
Testing and Rolling Out Enterprise Applications
Gem WeBlog
 
4 sdlc and stlc
Chandra Maddigapu
 
Software requirement enginering
Wajid Ali
 
Good Practices For Developing User Requirements
nkaur
 
Requirement analysis with use case
Rapeepan Thawornwanchai
 
Software Requirements and Specifications
vustudent1
 
System requirements engineering
Animesh Chaturvedi
 
testing
nazeer pasha
 
Sdlc
nazeer pasha
 
Software Engineering concept
Atamjitsingh92
 
Writing Effective Use Cases
Harsh Jegadeesan
 
Modern Elicitation Process
Rajon
 
Sdlc process document
Pesara Swamy
 
Requirements Engineering Processes
Ra'Fat Al-Msie'deen
 
Software development life cycle by chitta
Chittaranjan Das
 
Sdlc tutorial
Suresh Mca
 
Software Requirements (3rd Edition) summary
Ahmed Kamel Taha
 
Object oriented-systems-development-life-cycle ppt
Kunal Kishor Nirala
 
software engineering
Snow Queenzz
 
Ad

Viewers also liked (9)

PPTX
Sys ml helperprofile-rhapsody813-obtainandinstall-v1
Fraser Chadburn
 
PPTX
Software Engineering Methodology
Rajandeep Gill
 
PPTX
Teknik pemikiran kreatif
Mazmon Mahmud
 
PPTX
Survey research
sudha pandeya/pathak
 
PDF
Basics of surveying
Thomas Phanuel
 
PPT
Survey research
Punjab University Lahore
 
PPTX
Surveying
GAURAV. H .TANDON
 
PPT
Presentation of surveying
Shaker Ullah
 
PPTX
6 basic steps of software development process
Riant Soft
 
Sys ml helperprofile-rhapsody813-obtainandinstall-v1
Fraser Chadburn
 
Software Engineering Methodology
Rajandeep Gill
 
Teknik pemikiran kreatif
Mazmon Mahmud
 
Survey research
sudha pandeya/pathak
 
Basics of surveying
Thomas Phanuel
 
Survey research
Punjab University Lahore
 
Surveying
GAURAV. H .TANDON
 
Presentation of surveying
Shaker Ullah
 
6 basic steps of software development process
Riant Soft
 
Ad

Similar to Survey on Requirements Engineering Tools (20)

PDF
Rm tools
Anurag Tiwari
 
PPTX
Requirement engineering evaluation
Ishraq Al Fataftah
 
PDF
Component Based Software OW2 Conference Nov10
OW2
 
PDF
The Social Requirements Engineering (SRE) Approach to Developing a Large-scal...
Ralf Klamma
 
PDF
A Framework for Classifying and Comparing Architecture-Centric Software Evolu...
Pooyan Jamshidi
 
PPTX
The ROLE Requirements Bazaar
Dominik Renzel
 
PPTX
Requirements Engineering Essentials.pptx
blltariq21
 
PPTX
Chapter 1 - Requirement Engineering
Neeraj Kumar Singh
 
PPTX
Is&d
subhashishbehera
 
PPTX
Lecture 4
Ahmed Alageed
 
PPTX
Lecture4
Ahmed Alageed
 
PDF
RQS - Requirements Quality Suite
The REUSE Company
 
PPT
Requirements engineering vii
indrisrozas
 
PDF
Software Engineering 10th Edition Sommerville Solutions Manual
lumbuabobi
 
PDF
Download Software Engineering 10th Edition Sommerville Solutions Manual ebook...
rateebqalya
 
PPT
Requirements engineering i
indrisrozas
 
PDF
Agile and Requirements Trends & Benchmarks 2012 (Englisch)
SwissQ Consulting AG
 
PPTX
Modern Requirements Tools: Advantages & Applications
pd7.group
 
PPT
Analysis concepts and principles
saurabhshertukde
 
PPT
Week 1 lecture 2 - introduction to re
Hoo Jason
 
Rm tools
Anurag Tiwari
 
Requirement engineering evaluation
Ishraq Al Fataftah
 
Component Based Software OW2 Conference Nov10
OW2
 
The Social Requirements Engineering (SRE) Approach to Developing a Large-scal...
Ralf Klamma
 
A Framework for Classifying and Comparing Architecture-Centric Software Evolu...
Pooyan Jamshidi
 
The ROLE Requirements Bazaar
Dominik Renzel
 
Requirements Engineering Essentials.pptx
blltariq21
 
Chapter 1 - Requirement Engineering
Neeraj Kumar Singh
 
Is&d
subhashishbehera
 
Lecture 4
Ahmed Alageed
 
Lecture4
Ahmed Alageed
 
RQS - Requirements Quality Suite
The REUSE Company
 
Requirements engineering vii
indrisrozas
 
Software Engineering 10th Edition Sommerville Solutions Manual
lumbuabobi
 
Download Software Engineering 10th Edition Sommerville Solutions Manual ebook...
rateebqalya
 
Requirements engineering i
indrisrozas
 
Agile and Requirements Trends & Benchmarks 2012 (Englisch)
SwissQ Consulting AG
 
Modern Requirements Tools: Advantages & Applications
pd7.group
 
Analysis concepts and principles
saurabhshertukde
 
Week 1 lecture 2 - introduction to re
Hoo Jason
 

Recently uploaded (20)

PDF
Outsourced Audit & Assurance in USA Why Globus Finanza is Your Trusted Choice
Globus finanza
 
PDF
Power and position in leadershipDOC-20250808-WA0011..pdf
meghavinikumar2006
 
PDF
Business model innovation report 2022.pdf
pradvapal
 
PDF
COST SHEET- Tender and Quotation unit 2.pdf
MANJU N
 
PDF
MSPs in 10 Words - Created by US MSP Network
Mayur Gudka
 
PDF
DOC-20250806-WA0002._20250806_112011_0000.pdf
dhanusriss08
 
PDF
WRN_Investor_Presentation_August 2025.pdf
cmagee4
 
PPTX
Belch_12e_PPT_Ch18_Accessible_university.pptx
Haitham327103
 
PDF
A Brief Introduction About Julia Allison
Julia Allison
 
PPTX
Amazon (Business Studies) management studies
AbhirupVerma
 
PDF
Ôn tập tiếng anh trong kinh doanh nâng cao
thaoonguyenn2311
 
PPTX
The Marketing Journey - Tracey Phillips - Marketing Matters 7-2025.pptx
shannonzipoy2
 
DOCX
unit 1 COST ACCOUNTING AND COST SHEET
MANJU N
 
PDF
Unit 1 Cost Accounting - Cost sheet
MANJU N
 
PDF
Training And Development of Employee .pdf
nivethavm864
 
PDF
Elevate Cleaning Efficiency Using Tallfly Hair Remover Roller Factory Expertise
farsookmon97766765
 
PDF
Katrina Stoneking: Shaking Up the Alcohol Beverage Industry
CIO WOMEN MAGAIZNE CIO WOMEN MAGAIZNE
 
PPTX
Lecture (1)-Introduction.pptx business communication
HamzaGhani10
 
PDF
pdfcoffee.com-opt-b1plus-sb-answers.pdfvi
thaoonguyenn2311
 
PDF
20250805_A. Stotz All Weather Strategy - Performance review July 2025.pdf
FINNOMENAMarketing
 
Outsourced Audit & Assurance in USA Why Globus Finanza is Your Trusted Choice
Globus finanza
 
Power and position in leadershipDOC-20250808-WA0011..pdf
meghavinikumar2006
 
Business model innovation report 2022.pdf
pradvapal
 
COST SHEET- Tender and Quotation unit 2.pdf
MANJU N
 
MSPs in 10 Words - Created by US MSP Network
Mayur Gudka
 
DOC-20250806-WA0002._20250806_112011_0000.pdf
dhanusriss08
 
WRN_Investor_Presentation_August 2025.pdf
cmagee4
 
Belch_12e_PPT_Ch18_Accessible_university.pptx
Haitham327103
 
A Brief Introduction About Julia Allison
Julia Allison
 
Amazon (Business Studies) management studies
AbhirupVerma
 
Ôn tập tiếng anh trong kinh doanh nâng cao
thaoonguyenn2311
 
The Marketing Journey - Tracey Phillips - Marketing Matters 7-2025.pptx
shannonzipoy2
 
unit 1 COST ACCOUNTING AND COST SHEET
MANJU N
 
Unit 1 Cost Accounting - Cost sheet
MANJU N
 
Training And Development of Employee .pdf
nivethavm864
 
Elevate Cleaning Efficiency Using Tallfly Hair Remover Roller Factory Expertise
farsookmon97766765
 
Katrina Stoneking: Shaking Up the Alcohol Beverage Industry
CIO WOMEN MAGAIZNE CIO WOMEN MAGAIZNE
 
Lecture (1)-Introduction.pptx business communication
HamzaGhani10
 
pdfcoffee.com-opt-b1plus-sb-answers.pdfvi
thaoonguyenn2311
 
20250805_A. Stotz All Weather Strategy - Performance review July 2025.pdf
FINNOMENAMarketing
 

Survey on Requirements Engineering Tools

  • 1. Survey on Requirements Engineering Tools 1 / 26 Survey on Requirements Engineering Tools REFSQ Industry Track 2011 Juan M. Carrillo de Gea Christof Ebert jmcdg1@um.es, Vector, Stuttgart, Germany Joaquín Nicolás, José L. Fernández Alemán, Aurora Vizcaíno Ambrosio Toval Universidad de Castilla-La Universidad de Murcia, Spain Mancha, Spain Essen, Germany, March 29, 2011
  • 2. Survey on Requirements Engineering Tools 2 / 26 Contents 1 Introduction 2 Related work 3 Classification framework for RE tools’ capabilities 4 Research methodology Research goals Instrumentation Experimental procedure 5 Results Participants Experimental results Discussion Threats to validity 6 Conclusions and future work Conclusions Future work
  • 3. Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
  • 4. Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
  • 5. Survey on Requirements Engineering Tools 3 / 26 Introduction Introduction Benefits from automated support to RE RE tools and support to the RE process RE tools and developer’s expectations What are the RE tools’ desirable features? How are they supported by current RE tools? Description of the state-of-the-art on RE tools Current RE tools identification Framework selection and adaptation Survey conduction Updated overview on RE tools’ capabilities and potentials
  • 6. Survey on Requirements Engineering Tools 4 / 26 Related work Related work Surveys on RE tools Surveys on RE [Zowghi and Coulin, 2005] [Liu et al., 2010] [Alenljung and Persson, 2008] [Winkler and von Pilgrim, 2010] [Hall, 2008] [Carlshamre et al., 2001] [Gregoriades and Sutcliffe, 2005] [Johansson et al., 2001] [Maiden et al., 2006] [Benslimane et al., 2007] [Portillo Rodríguez et al., 2010]
  • 7. Survey on Requirements Engineering Tools 5 / 26 Classification framework for RE tools’ capabilities Regulations and guidelines ISO/IEC TR 24766:2009 Technical Report (TR) of Type 2 Set of capabilities that the RE tools should support Six major categories Requirements elicitation Requirements analysis Requirements specification Requirements V&V Requirements management Other tool capabilities Supplements ISO/IEC 14102:2008 International Standard Evaluation of CASE tools
  • 8. Survey on Requirements Engineering Tools 6 / 26 Classification framework for RE tools’ capabilities RE tools’ capabilities ISO/IEC TR 24766 tool capabilities Category Amount Requirements elicitation 37 Requirements analysis 36 Requirements specification 16 Requirements verification and validation 34 Requirements management 17 Other tool capabilities 17 Total 157
  • 9. Survey on Requirements Engineering Tools 7 / 26 Research methodology Research goals Goal/Question Metric (GQM) framework Goal/Question Metric (GQM) framework [Basili and Rombach, 1988] GQM template [Basili et al., 1999] Goal: To investigate the state-of-the-art on RE tools using a questionnaire aimed at software vendors Question: Do current RE tools address industry challenges? Metric: Capabilities of RE tools (questionnaire scores)
  • 10. Survey on Requirements Engineering Tools 8 / 26 Research methodology Instrumentation Instrumentation I Databases hosting RE tools lists (July, 2010–August, 2010) Database Amount Ian Alexander 67 Alarcos Research Group 7 INCOSE 34 Ludwig Consulting Services 40 Qaguild 7 Volere 71 @WEBO 41 Total sample size after discarding invalid tools 94
  • 11. Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
  • 12. Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
  • 13. Survey on Requirements Engineering Tools 9 / 26 Research methodology Instrumentation Instrumentation II DESMET [Kitchenham, 1996] method for evaluating software engineering methods and tools Feature Analysis—Survey 146-items questionnaire 6 ISO TR 24766 categories of features plus modelling plus traceability 157 ISO TR 24766 features → 126 questions 126 technical questions + 20 general, administrative questions Web-based survey using LimeSurvey
  • 14. Survey on Requirements Engineering Tools 10 / 26 Research methodology Experimental procedure Experimental procedure Questionnaire and survey system preparation: September, 2010–November, 2010 Tool representatives filled in the survey: December 20, 2010–January 15, 2011 Follow-up email and deadline extension to February 7, 2011
  • 15. Survey on Requirements Engineering Tools 11 / 26 Results Participants Participants I 38 participants out of 94 candidates invited (40.42%) Acclaro DFSS Leap SE CASE Spec Aligned Elements MacA&D/WinA&D Cognition Cockpit Avenqo PEP MKS Integrity Cradle Blueprint PACE G-MARC Bright Green Projects Polarion inteGREAT Caliber RM Requirements IRQA Cameo Psoda jUCMNav Requirements+ QFDcapture
  • 16. Survey on Requirements Engineering Tools 12 / 26 Results Participants Participants II QPack RTIME TestTrack RM RaQuest RequisitePro TopTeam Analyst Rational DOORS RMTrak TraceCloud ReqMan Rommana TrackStudio Reqtify Scenario Plus VisibleThread On- Requirements premise/On-demand SpiraTeam Composer
  • 17. Survey on Requirements Engineering Tools 13 / 26 Results Experimental results Administrative information I Year of first release Year of last release 8 30 7 25 6 5 20 4 15 3 10 2 1 5 0 0 1975 1977 1979 1981 1983 1985 1987 1989 1991 1993 1995 1997 1999 2001 2003 2005 2007 2009 2011 2008 2009 2010 2011 No. of tools No. of tools
  • 18. Survey on Requirements Engineering Tools 14 / 26 Results Experimental results Administrative information II Platform required Type of license Windows/ Windows/ Windows/ Open- Open- Mac OS Linux Mac source and source and OS/Linux free not-free Windows/ Proprietary Mac OS/ and free UNIX/Linux Windows/ UNIX/Linux Windows Web-based Proprietary and not- free
  • 19. Survey on Requirements Engineering Tools 15 / 26 Results Experimental results Administrative information III Cost per individual license Amount of licenses in use 18 14 16 12 14 10 12 10 8 8 6 6 4 4 2 2 0 0 No answer Less than 100 to 500 501 to More than No answer 0 to 100 101 to 1001 to More than 100 1000 1000 1000 10000 10000 Cost No. of licenses
  • 20. Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
  • 21. Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
  • 22. Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
  • 23. Survey on Requirements Engineering Tools 16 / 26 Results Experimental results Correlation between variables 11 variables—one for each category of features plus global score plus cost per individual license plus amount of licenses in use Bivariate correlation tests (Pearson’s Correlation Coefficients) Correlation is significant at the (**) 0.01 level/(*) 0.05 level (1-tailed) Strong direct correlation between each distinct category of features Cost per individual license–analysis (0.336*), modelling (0.404*), traceability (0.329*), specification (0.545**) and global score (0.358) Number of licenses in use–other tool capabilities (0.513**), cost per individual license (0.243) and global score (0.183)
  • 24. Survey on Requirements Engineering Tools 17 / 26 Results Experimental results Technical information I Tools’ scores 20 18 18 17 17 16 16 14 14 1212 12 12 12 11 11 10 10 10 9 8 8 7 6 6 6 6 5 5 5 5 4 4 4 4 4 3 3 3 2 2 2 1 1 1 1 1 0 0 0 0 Very high High Medium Low Very low Elicitation Analysis Specification Modelling V&V Management Traceability Other tool capabilities
  • 25. Survey on Requirements Engineering Tools 18 / 26 Results Experimental results Technical information II Global score (level of accomplishment of the entire ISO/IEC TR 24766)—only calculated for those tools participating in all categories 12 10 8 6 4 2 0 Very high High Medium Low Very low Global
  • 26. Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
  • 27. Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
  • 28. Survey on Requirements Engineering Tools 19 / 26 Results Discussion Scenario 1: Requirements elicitation and V&V Storing and managing templates for elicitation (57%), elicitation checklists (60%), prioritization forms (57%) Providing OMG ReqIF/RIF compatibility (26%) Generating exception reports on verification/validation plan cases ↔ verification/validation procedures (57%/60%) Providing standard format for interfacing to verification/validation tools (57%/57%) Implementing some built-in requirements checks (50%) Both: Cockpit, Cradle, QPack, Reqtify Elicitation: MKS Integrity, Polarion Requirements V&V: Aligned Elements, CASE Spec, G-MARC, IRQA, PACE, ReqMan, TraceCloud
  • 29. Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
  • 30. Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
  • 31. Survey on Requirements Engineering Tools 20 / 26 Results Discussion Scenario 2: Requirements modelling and specification Providing storage and display of BPMN (44%), goal models (39%), SysML artifacts (31%), DFDs (44%) Checking the document through spell checking, grammar checking, data dictionaries, and acronym tables (60%) Generating the output of the specification in a finished form (65%) Complete loop between RE tool and formatted document (39%) Both: Cockpit, Cradle, PACE Modelling: Reqtify Specification: G-MARC, inteGREAT, MKS Integrity, QPack, DOORS, TraceCloud, VisibleThread
  • 32. Survey on Requirements Engineering Tools 21 / 26 Results Discussion Scenario 3: Requirements traceability Generating reports that compare current and previous versions when a source document is updated (55%) Tracing across the tools’ boundaries (55%) Tracing text to graphics (52%), graphics to graphics (47%), elements within graphics (42%), tables and cells within a table (36%) Generating reports of traceability attributes (60%) Best: CASE Spec, Cockpit, Cradle, G-MARC, inteGREAT, Reqtify Close to the best: Avenqo PEP, IRQA, Polarion Requirements, QPack, TopTeam Analyst
  • 33. Survey on Requirements Engineering Tools 21 / 26 Results Discussion Scenario 3: Requirements traceability Generating reports that compare current and previous versions when a source document is updated (55%) Tracing across the tools’ boundaries (55%) Tracing text to graphics (52%), graphics to graphics (47%), elements within graphics (42%), tables and cells within a table (36%) Generating reports of traceability attributes (60%) Best: CASE Spec, Cockpit, Cradle, G-MARC, inteGREAT, Reqtify Close to the best: Avenqo PEP, IRQA, Polarion Requirements, QPack, TopTeam Analyst
  • 34. Survey on Requirements Engineering Tools 22 / 26 Results Discussion Scenario 4: Other features Providing Eclipse support (42%) Providing data federation (28%) Providing an open data model (18%)
  • 35. Survey on Requirements Engineering Tools 23 / 26 Results Threats to validity Threats to validity Internal validity (causal relationships) Mortality percentage (7.32%) Commitment of the RE tools representatives Truthfulness of the answers (Hawthorne effect) Triangulation technique Questionnaire planning and design External validity (generalizations) Participants are representative of the RE tools’ community
  • 36. Survey on Requirements Engineering Tools 23 / 26 Results Threats to validity Threats to validity Internal validity (causal relationships) Mortality percentage (7.32%) Commitment of the RE tools representatives Truthfulness of the answers (Hawthorne effect) Triangulation technique Questionnaire planning and design External validity (generalizations) Participants are representative of the RE tools’ community
  • 37. Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
  • 38. Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
  • 39. Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
  • 40. Survey on Requirements Engineering Tools 24 / 26 Conclusions and future work Conclusions Conclusions The RE process is well-covered by current RE tools Connection between the scores accomplished by the RE tools in each category of features More expensive tools offer better requirements analysis, specification, modelling and traceability support No association between the cost per individual license and the global score More extended tools are stronger in ISO/IEC TR 24766 other tool capabilities The amount of licenses in use is not associated with the cost per individual license nor with the global score
  • 41. Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
  • 42. Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
  • 43. Survey on Requirements Engineering Tools 25 / 26 Conclusions and future work Future work Future work Support to GSD settings Concrete RE capabilities Elicitation Modelling Traceability ... Dissemination of results
  • 44. Survey on Requirements Engineering Tools 26 / 26 Thank you for your attention Thank you! Any questions?
  • 45. Survey on Requirements Engineering Tools 26 / 26 Thank you for your attention Thank you! Any questions?
  • 46. Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
  • 47. Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
  • 48. Survey on Requirements Engineering Tools 27 / 26 Additional information Score calculation Participation of the tool t in the category c: true, NA(t , c) ≥ 0.5 · NQ(c); participant (t , c) = false, otherwise. NA(t , c): number of answers of the tool t in the category c NQ(c): number of questions of the category c Score s of the tool t in the category c: NQ(c) q=1 score(t ,q) ·4 score(t , c) = NQ(c) score(t , q) ∈ {0, 1}: score of the t in the question q tool  Very low, s ∈ [0, 0.5];   Low, s ∈ (0.5, 1.5];   ∀s ∈ [0, 4], discretisation(s) = Medium, s ∈ (1.5, 2.5];  High, s ∈ (2.5, 3.5];    Very high, s ∈ (3.5, 4]. 
  • 49. Survey on Requirements Engineering Tools 28 / 26 Bibliography Bibliography I Alenljung, B. and Persson, A. (2008). DESCRY: a method for evaluating decision-supporting capabilities of requirements engineering tools. In Proc. of the 14th Int. Conf. on Requir. Eng.: Foundation for Software Quality, pages 52–57. Springer-Verlag. Basili, V. and Rombach, H. (1988). The TAME project: towards improvement-oriented software environments. IEEE Trans. Softw. Eng., 14:758–773. Basili, V. R., Shull, F., and Lanubile, F. (1999). Building knowledge through families of experiments. IEEE Trans. Softw. Eng., 25:456–473.
  • 50. Survey on Requirements Engineering Tools 29 / 26 Bibliography Bibliography II Benslimane, Y., Cysneiros, L. M., and Bahli, B. (2007). Assessing critical functional and non-functional requirements for web-based procurement systems: a comprehensive survey. Requir. Eng., 12:191–198. Carlshamre, P., Sandahl, K., Lindvall, M., Regnell, B., and Dag, J. N. (2001). An industrial survey of requirements interdependencies in software product release planning. In Proc. of the Fifth IEEE Int. Symp. on Requir. Eng., pages 84–92, Washington, DC, USA. IEEE Computer Society. Gregoriades, A. and Sutcliffe, A. (2005). Scenario-based assessment of nonfunctional requirements. IEEE Trans. Softw. Eng., 31:392–409.
  • 51. Survey on Requirements Engineering Tools 30 / 26 Bibliography Bibliography III Hall, R. J. (2008). A method and tools for large scale scenarios. Autom. Softw. Eng., 15:113–148. Johansson, E., Wesslén, A., Bratthall, L., and Höst, M. (2001). The importance of quality requirements in software platform development - a survey. In Proc. of the 34th Annual Hawaii Int. Conf. on Sys. Sci., volume 9, Washington, DC, USA. IEEE Computer Society. Kitchenham, B. (1996). DESMET: a method for evaluating software engineering methods and tools. Tech. Report TR96-09, Dept. of Computer Science, University of Keele.
  • 52. Survey on Requirements Engineering Tools 31 / 26 Bibliography Bibliography IV Liu, L., Li, T., and Peng, F. (2010). Why requirements engineering fails: a survey report from China. In Proc. of the 2010 18th IEEE Int. Requir. Eng. Conf., pages 317–322. IEEE Computer Society. Maiden, N., Seyff, N., Grunbacher, P., Otojare, O., and Mitteregger, K. (2006). Making mobile requirements engineering tools usable and useful. In Proc. of the 14th IEEE Int. Requir. Eng. Conf., pages 26–35. Portillo Rodríguez, J., Ebert, C., and Vizcaíno, A. (2010). Technologies and tools for distributed teams. IEEE Softw., 27:10–14. Winkler, S. and von Pilgrim, J. (2010). A survey of traceability in requirements engineering and model-driven development. Softw. and Sys. Modeling, 9:529–565.
  • 53. Survey on Requirements Engineering Tools 32 / 26 Bibliography Bibliography V Zowghi, D. and Coulin, C. (2005). Requirements elicitation: a survey of techniques, approaches, and tools. In Aurum, A. and Wohlin, C., editors, Engineering and Managing Software Requirements, pages 19–46. Springer.