The Quamoco Quality Modelling and Assessment Approach

Editor's Notes

• #2: I‘m delighted to present to you the results on quality modelling and assessment of our three year research project Quamoco.\nI‘ve always found quality an interesting concept because it determines so much about a system, but also it is very complex.\n
• #3: As David Garvin pointed out: Quality is complex and multifaceted and therefore it is also the source of great confusion“.\nSo what do computer scientists do to handle complexity? They abstract! Researchers have developed a variety of software quality models.\n\n
• #4: The range of quality models goes from collections of metrics over academic models, domain standards to company-specific models. But none of them has been able to get really broad acceptance. When we set out to work on quality assessments, we wanted to find a good basis. What do you do when there is no clear leader? You looked at the ISO standard, here 9126.\nISO 15005: Road vehicles - Ergonomic aspects of transport information and control systems - Dialogue management principles and compliance procedures\nNHPP: Non-homogeneous Poisson process (reliability growth models)\n
• #5: It breaks down quality into quality attributes such as reliability or maintainability – the „-ilities“. It then breaks them further down and gives some metrics to measure them.\nHow well is it doing in practice? We asked over a hundred practicioners.\n
• #6: And the result is not pretty good.\nOnly 28% of the respondents of our international survey said that they use ISO 9126. Only 28%. A bit more than a quarter.\nWhy is that so? We asked in detailed interviews about the reasons.\n
• #7: The developers told us taht there is a huge gap between the abstract quality attributes of ISO 9126 and the concrete implementation and assessment on the product. Operationalising the quality attributes is considered extremely difficult. Hence, they use some metrics.\nThe existing metrics are concrete but lack a clear connection to quality goals.\n
• #8: Hence, we have the abstract quality attributes of ISO 9126 or similarly the new standard 25010 as well as various measures.\n
• #9: And there is this gap that prevents quality attributes from being assessed and measures from clearly contributing to quality goals.\n
• #10: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #11: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #12: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #13: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #14: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #15: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #16: The Quamoco project has worked three years on providing – among other things – four results to help to overcome this problems.\n
• #17: \n
• #18: \n
• #19: Schließen der Lücke, durchgängie Zusammenhänge\n
• #20: Beispiel\n
• #21: The higher in this quality model, the more general the model should be. So on the top level, we have ISO 25010 quality attribute, which are almost applicable to all software products.\nThe product factors should be quite generally applicable but on the measure level, we are often specific for technologies or languages. To have general measures and to decouple them from technical implementations, we introduced instruments.\n
• #22: For example, we have a product factor „Uselessness of Methods“, which we measure, for example, with „Statically unused method“. This measure is applicable to different languages. Hence, we refine this with an instrument that uses Gendarme for C# and PMD for Java.\n
• #23: Base model describes qualities important for almost any kind of software\nAim is to use it as basis for more specific quality models and also to be able to apply it directly for very popular paradigms and technologies\nIt has a modular structure\nroot contains quality attributes and very general product factors, modules for object-oriented factors and operationalisation for Java and C#\nmostly static analysis tools and inspection\nprototypical development for C, C++ and GUI\n 284 Faktoren\n 524 Maße\n
• #24: \n
• #25: With that, we can do an actual assessment or evaluation of a software product.\nWe measure using the instruments and collect values for all the measures.\nNow, we need evaluations and aggregations for the product factors and quality attributes.\n
• #26: With that, we can do an actual assessment or evaluation of a software product.\nWe measure using the instruments and collect values for all the measures.\nNow, we need evaluations and aggregations for the product factors and quality attributes.\n
• #27: \n
• #28: \n
• #29: \n
• #30: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #31: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #32: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #33: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #34: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #35: From over a hundred systems, found typical distributions, eliminated outliers and analysed quartiles\nThe result of an evaluation function is a value between 0 and one.\n
• #36: The model was a dictation in school, if you have some errors you will get a bad grade\nHere: German school grades but you could plug in any interpretation wodel\n
• #37: \n
• #38: \n
• #39: \n
• #40: \n
• #41: \n
• #42: Eine solche Bewertung muss valide sein, um Entscheidungen darauf abzustützen. Haben wir eine gute Aussage über das System gemacht?\nHier haben wir unsere Bewertungen mit Expertenbewertungen verglichen.\nEs ergab sich fast exakt die gleiche Rangfolge. Wir waren in dieser Auflösung also bereits so gut wie Experten.\n
• #43: Hier für fünf Subsysteme eines kommerziellen Systems für den Maintainability-Teil.\n
• #44: \n
• #45: \n
• #46: \n
• #47: \n
• #48: \n
• #49: \n
• #50: \n
• #51: \n
• #52: \n
• #53: \n
• #54: \n
• #55: \n
• #56: And with that I want to close with a a thanks to all the Quamoco partners and supporters!\n
• #57: \n