Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement
1 like489 views
The document presents the spago4q open-source platform and the qest nd model for software performance measurement, emphasizing the integration of performance evaluation with business intelligence tools. It outlines a structured methodology for assessing software projects through a five-step process, utilizing key performance indicators (KPIs) to analyze and improve productivity. Additionally, a case study demonstrates the practical application of the model, highlighting its effectiveness in multi-dimensional performance evaluation.
Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement
- 1. Spago4Q and the QEST nD Model: An Open Source Solution for Software Performance Measurement Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Mauro Regoli Dipartimento di Tecnologie dell’Informazione Università degli Studi di Milano – Italy Sergio Oltolina, Gabriele Ruffatti Engineering Ingegneria Informatica - Italy
- 2. Outline • QEST nD Model • Spago4Q Open Source Platform • Roadmap to build the Integrated Framework • Case Study OSS 2010 2
- 3. Introduction • The availability of detailed and updated information on development process is paramount – To maintain the competitiveness level – To operate in new and more challenging markets • The scenario of integrated information is known as Business Intelligence, and enclose all the processes and tools for data acquisition and analysis • A number of structured process models have been adopted depending on the domain and size – To collect specific knowledge about their development processes – To strengthen the know-how in terms of more efficiency and quality OSS 2010 3
- 4. Goals • Deploy an environment that integrates models for software performance evaluation in a more complete and coherent framework – Non-invasive – Dynamic – Configurable – Scalable – Based on a formal mathematical model • Provide a solution integrating – The QEST nD model for process performance evaluation – The Open Source Business Intelligence platform Spago4Q OSS 2010 4
- 5. The QEST nD Model - 1 • Several mono-dimensional performance models are available in SE context that integrates individual measurements into a single performance index – Models are oversimplified and unable to reflect different performance aspects • Traditional domains (e.g., Business Modeling) exploit multi- dimensional models – Data derived directly from accounting systems, implying multiple analysis viewpoints – Complex viewpoints analyze distinct aspects of the overall process performance OSS 2010 5
- 6. The QEST nD Model - 2 • The QEST nD model is aimed at measuring software project performances addressing multi-dimensionality and qualitative-quantitative assessment • In the QEST model the quality can be viewed as the integration of at least three different viewpoints – Economical – Social – Technical • User-specific dimensions could be defined to evaluate ad-hoc analysis dimensions L. Buglione, “Misurare il software. Quantità, Qualità, Standards e Miglioramento di processo nell'Information & Communication Technology,” 3rd Edition, Franco Angeli, 2008 OSS 2010 6
- 7. Mathematical Formalization • QEST Performance value (P) given by the combination – Quantitative measurements (RP-Rough Productivity) – Qualitative measurements, calculated as a perception-based measurement of the overall product quality (QF-Quality Factor) OSS 2010 7
- 8. Mathematical Formalization - 2 • The basic 3-D model QEST is represented as a geometrical regular tetrahedron – Corners E, S, T and the convergence of the edges to the P vertex describe the performance level – Points Qe, Qs, Qt represent the respective dimension values (Rough Productivity) • Values calculated as the weighted sum of a list of n distinct measures • Represented as the distance between points and corners • If Quality factor is applied, points Q’e, Q’s, Q’t represent an upward or downward translation of Qe, Qs, Qt OSS 2010 8
- 9. Mathematical Formalization - 3 • Performance P calculated as the distance between the center of gravity of the original tetrahedron and the center of the plane (Q’e, Q’s, Q’t) section along the tetrahedron height • Through computational geometry, the multi-dimensional case is described with a simple formula for global performance value computation OSS 2010 9
- 10. Spago4Q • Spago4Q (SpagoBI for Quality) is an open source platform for the continuous monitoring of software quality – Multi-process multi-project monitoring – Collection of data executed in a fully-transparent way – Equipped with extractors specific of most-common software process environments (IDE, workflow management, text editing, …) • Spago4Q (www.spago4q.org) is a vertical adaptation of SpagoBI, the Open Source Business Intelligence suite (www.spagobi.org) • Recently adopted by European Commission – Directorate General for Regional Policy OSS 2010 10
- 11. Spago4Q Structure OSS 2010 11
- 12. Spago4Q Metamodel OSS 2010 12
- 13. An Integrated Environment • The definition of a QEST nD model is a five-step procedure fully supported by Spago4Q • The procedure is coherent with the PMAI (Plan-Measure- Assess-Improve) cycle: – PLAN, defining a set of KPIs, basing on the GQM approach, and dimensions that characterize the analysis – MEASURE, which includes the collection of data, and the computation of KPI values and global performance value – ASSESS, where results are presented in dashboards and reports – IMPROVE, where every negative or low value is deeply analyzed to find problems in the process OSS 2010 13
- 14. 1st Step: KPIs and Model Definition • Declaration of a complete GQM, with the definition of – the analysis dimensions – the concepts to measure – the KPIs to apply to project work-products OSS 2010 14
- 15. 2nd Step: Weights and Thresholds Definition • Couple each KPI with the respective weight – Indicates the importance that such a concept plays in the dimension it belongs to • Define the specific thresholds – Evaluates the value with respect to organization policies • Assign (if provided) the QF to each specific dimension OSS 2010 15
- 16. 3rd Step: Value Collection • Measures are taken directly from Spago4Q data warehouse – The DB is filled by data automatically collected by extractors accessing process work-products (code package, text documents, project information, …) • KPIs are described in terms of – Default value – Minimum and maximum values (for normalization) – KPI computation algorithm OSS 2010 16
- 17. 4th Step: Global Performance Computation • Global and dimension-wise performance indexes are computed as KPIs that take in input configuration data and results of the KPIs • Dimension performance is calculated as the weighted sums of each specific KPI OSS 2010 17
- 18. 5th Step: Reports • Sets of reports and dashboards could be defined and configured to satisfy reporting and managerial needs • Spago4Q provides methods and interfaces to directly configure and create new reports using all the facilities provided by the open source reporting tool Eclipse BIRT OSS 2010 18
- 19. Case Studies • Case study based on the monitoring of the complex performance of three real projects with several measurements in a three-month time slot • We use a QEST nD model called Business-Service Model and taking into consideration four specific analysis dimensions: 1. QEST-EC: Economic performance indicator 2. QEST-RS: Resource performance indicator 3. QEST-TE: Technical performance indicator 4. QEST-CS: Customer Satisfaction performance indicator OSS 2010 19
- 20. Case Studies: GQM OSS 2010 20
- 21. Case Studies: KPIs OSS 2010 21
- 22. Case Studies: Dashboards • Spago4Q dashboard gives an immediate snapshot of the situation, highlighting problems and suggesting areas that need a process quality improvement – Red squares indicate under-threshold performance values OSS 2010 22
- 23. Spago4Q: Reports • Reports indicate that all projects were concluded with an excellent global performance (close to one), showing some issues in the process that is worth analyzing – Project3 experienced bad performances in Technological and Economic dimensions OSS 2010 23
- 24. Conclusions • We provide a complete multi-project multi-process performance evaluation environment combining – The mathematical formalization of the QEST nD model – The monitoring and reporting facilities offered by Spago4Q • It gives the possibility to analyze the performance of the development process from different points of view integrating semantically different KPIs in a single indicator • The described QEST nD model will be made available in the future version of Spago4Q (www.spago4q.org) OSS 2010 24
- 25. Questions? Thank you for the attention OSS 2010 25