Using the Machine to predict Testability
3 likes724 views
This document discusses using machine learning to predict testability based on source code metrics. It begins with an introduction to the presenting organization and definitions of testability and machine learning concepts. It then shows how decision trees and other machine learning approaches could be used to predict testability levels (high, medium, low) based on source code metrics like number of interfaces, abstractness, and coupling. As an example, metrics from 9 Java packages were analyzed to build and test a predictive model in the Weka machine learning software. However, the document notes the initial model is simplistic and could be improved by incorporating more metrics related to factors in the testability fishbone diagram.
![TestabilityTestability as a Set of Factors.ISO defines testability as “attributes of software that bear on the effort needed to validate the software product” [ISO/IEC 9126].Various factors can contribute to testability (obvious).Which are the factors?](https://image.slidesharecdn.com/btd2011-110217030308-phpapp01/85/Using-the-Machine-to-predict-Testability-10-320.jpg)
![Testability Fish-Bone by [Binder].Testability](https://image.slidesharecdn.com/btd2011-110217030308-phpapp01/85/Using-the-Machine-to-predict-Testability-11-320.jpg)
![Testability Factors [Blinder]Documentation: With regards to testing, requirements and specifications are of prime importance.Implementation: The implementation is the target of all testing, and thus the extent to which the implementation al- lows itself to be tested is a key factor of the testing effort.Testability](https://image.slidesharecdn.com/btd2011-110217030308-phpapp01/85/Using-the-Machine-to-predict-Testability-12-320.jpg)
![Testability Factors [Blinder]Test Suite: Factors of the test suite itself also determine the effort required to test. Desirable features of test suites are correctness, automated execution and reuse of test cases.Test Tools: The presence of appropriate test tools can alleviate many problems that originate in other parts of the ‘fish bone’ figure.Process Capability: The organizational structure, staff and resources supporting a certain activity are typically referred to collectively as a (business) process. Properties of the testing process obviously have great influence on the effort required to perform testing.Testability](https://image.slidesharecdn.com/btd2011-110217030308-phpapp01/85/Using-the-Machine-to-predict-Testability-13-320.jpg)
Using the Machine to predict Testability
- 1. Using the Machine to predict TestabilityMiguel Lopez - ALGORISMI
- 2. AgendaWho are we? Goal of the Presentation Definition of Testability Some Machine Learning ConceptsPredict TestabilityConclusionQuestions/Answers
- 3. Algorismi is the Innovative Software Quality Company.Funded by IRSIB for R&D ProjectsExperienced Team : Operation,Development and DeploymentInnovation and Efficiency (cost / time)Our mission is to help you toDeliverBetter Software
- 5. And to be prepared to change
- 6. By InnovatingWho are we?3
- 7. Some Aphorisms displayed on our Walls“The world is changing very fast. Big will not beat small anymore. It will be the fast beating the slow.” Rupert Murdoch, Chairman and CEO, News CorporationChange is inevitable, stability, and security a myth.Be prepared to change, to anticipate, provoke, participate ... but mostly avoid the subject.Who are we?4
- 8. Software Quality OfferX-TRAXOur Test Management tool.ScaleOur Code Auditing tool.SQA ServiceWho are we?5
- 9. Goal of the PresentationShow how to use Machine Learning Algorithms in the test management.Based on a toy example, we show how to proceed.Very small example of what we do in our R&D team.We want you to probe this in your company.Who are we?6
- 10. TestabilityTestability as a Set of Factors.ISO defines testability as “attributes of software that bear on the effort needed to validate the software product” [ISO/IEC 9126].Various factors can contribute to testability (obvious).Which are the factors?
- 11. Testability Fish-Bone by [Binder].Testability
- 12. Testability Factors [Blinder]Documentation: With regards to testing, requirements and specifications are of prime importance.Implementation: The implementation is the target of all testing, and thus the extent to which the implementation al- lows itself to be tested is a key factor of the testing effort.Testability
- 13. Testability Factors [Blinder]Test Suite: Factors of the test suite itself also determine the effort required to test. Desirable features of test suites are correctness, automated execution and reuse of test cases.Test Tools: The presence of appropriate test tools can alleviate many problems that originate in other parts of the ‘fish bone’ figure.Process Capability: The organizational structure, staff and resources supporting a certain activity are typically referred to collectively as a (business) process. Properties of the testing process obviously have great influence on the effort required to perform testing.Testability
- 14. Focus on Source Code FactorTestability
- 15. Some Heuristics for Testability Heuristic #1 ReuseFavor modularity before reuse. Its better to have code duplicates than to delay testing of a component because required changes to a superclass or library class it depends on are pending.Heuristic: Give higher priority to the modularity of a system than to the reuse of components.Testability
- 16. Some Heuristics for Testability Heuristic #2 Loose CouplingLoosely coupled software is one where each of its components has, or makes use of, little or no knowledge of the definitions of other separate components.Heuristic: Reduce the number of used classes.Testability
- 17. Some Heuristics for Testability Heuristics and Object-Oriented MetricsTestability
- 18. Some Heuristics for Testability Reuse - Number of InterfacesAn interface in the Java programming language is an abstract type.
- 19. Interfaces are declared using the interface keyword, and may only contain method signatures and constant declarations.
- 20. Interfaces cannot be instantiated. A class that implements an interface must implement all of the methods described in the interface.Testability
- 21. Some Heuristics for Testability Reuse - AbstractnessAbstractness = Na/Ncwhere Na = Number of abstract classes in a packageNc = Number of concrete classes in a package.Abstractness = 0 means a completely concrete package.
- 22. Easy to test.
- 23. Abstractness = 1 means a completely abstract package.Testability
- 24. Some Heuristics for Testability Loose Coupling Afferent Coupling between PackagesAfferent Coupling between packages (Ca) measures the total number of external classes coupled to classes of a package due to incoming coupling (coupling from classes external classes of the package, uses CBO definition of coupling).
- 25. Each class counts only once. Zero if the package does not contain any classes or if external classes do not use the package's classes. Testability
- 26. Some Heuristics for Testability Loose CouplingEfferent Coupling between PackagesEfferent Coupling between packages (Ce) measures the total number of external classes coupled to classes of a package due to outgoing coupling (coupling to classes external classes of the package, uses Ce definition of coupling).
- 27. Each class counts only once. Zero if the package does not contain any classes or if external classes are not used by the package's classes. Testability
- 28. Metric for TestabilityWe will use a qualitative metric for testability: Testability Level.
- 29. Ordinal scale:
- 30. High : Highly testable (easy to test)
- 31. Medium: normal effort to test.
- 32. Low: Lowly testable (difficult to test)
- 33. Testability is related to unit testing effort.Testability
- 34. What’s Machine Learning ?Machine learning, a branch of artificial intelligence, is a scientific discipline that is concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data.Data can be seen as examples that illustrate relations between observed variables.A major focus of machine learning research is to automatically learn to recognize complex patterns and make intelligent decisions based on data.Machine Learning
- 35. What’s Machine Learning ?Many approaches exist in the Machine Learning world.Neural networks, Bayesian networks, clustering,..Machine Learning
- 36. Some ML Approaches – Decision Tree Decision tree learning uses a decision tree as a predictive model which maps observations about an item to conclusions about the item's target value.Machine Learning
- 37. Some ML Approaches – Association Rule Association rule learning is a method for discovering interesting relations between variables in large databases.A typical and widely-used example of association rule mining is Market Basket Analysis.Example: Association rule "If A and B are purchased then C is purchased on the same trip"Machine Learning
- 38. Some ML Approaches – Neural Network An artificial neural network (ANN), usually called "neural network" (NN), is a mathematical model or computational model that tries to simulate the structure and/or functional aspects of biological neural networks. It consists of an interconnected group of artificial neurons and processes information using a connectionist approach to computation.They are usually used to model complex relationships between inputs and outputs or to find patterns in data.Example: Milan Lab: Soccer ClubPredict injuries of the soccer player.Machine Learning
- 39. Some ML Approaches – Clustering Cluster analysis or clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. Clustering is a method of unsupervised learning, and a common technique for statistical data analysis.Machine Learning
- 40. Some ML Approaches – Clustering A Bayesian network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional independencies via a directed acyclic graph (DAG). For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Machine Learning
- 41. Machine Learning & StatisticsStatistics: focus on understanding data in terms of modelsStatistics: interpretability, hypothesis testingMachine Learning: greater focus on predictionMachine Learning: focus on the analysis of learningalgorithms : not just large dataset.Machine Learning
- 42. Simple Example - WeatherIn the weather example, we want to predict the weather conditions to play outside.According to 4 variables (outlook,temperature, humidity, windy), we ask the following question:Can the children play in the garden?Machine Learning
- 43. Simple Example - WeatherMachine Learning
- 44. WEKA – ML SoftwareFor the Machine Learning, we use the software Weka.
- 45. Weka is an open source project of the Machine Learning Group at University of Waikato.
- 46. They have incorporated several standard ML techniques into a software "workbench" called WEKA, for Waikato Environment for Knowledge Analysis.
- 47. But what is a Weka?Machine Learning
- 48. WEKA – ML SoftwareFound only on the islands of New Zealand, the Weka is a endangered flightless bird.Machine Learning
- 49. Simple Example -WeatherTwo steps process:Train the machine: build the decision tree based upon a data set.Predict on new data set: load the decision tree and ask for a new data set.Question: Can the children play outside with the following weather conditions:Outlook: rainyTemperature: 59°FHumidity: 89Windy: falseMachine Learning
- 50. Simple Example -WeatherSee Demo WekaMachine Learning
- 51. Metrics for Testability Predict Testability
- 52. Metrics for Testability We will proceed the same way as in the weather example.
- 53. OO Metrics are the attributes used to predict.
- 54. Outlook, temperature, humidity, windy
- 55. Attribute to predict is testability.
- 57. Data for TestabilityWe analyzed 9 packages of Java Application (Scale).
- 58. For each package, we compute the different metrics with the Eclipse plugin Metrics.
- 60. Abstractness
- 63. For each package, we assess the testability level of the package.Predict Testability
- 64. Data for Testability: ARFF FormatPredict Testability
- 65. Simplistic ModelMetrics model is too simplistic.
- 66. Inheritance
- 68. …
- 69. The testability fish-bone must be measured.
- 70. Qualitative & Quantitative measures.Conclusion
- 71. Automate ML ProcessMeasurement Process must be automated.
- 72. Many Metrics Tools can be automated.
- 73. Learning & Prediction processes must be automated.
- 78. Bibliography[Blinder] R. Binder. Design for testability in object-oriented systems. Comm. of the ACM, 37(9):87–101, 1994.Predicting Class Testability using Object-Oriented Metrics, M. Bruntink and A. van Deursen.Data mining: practical machine learning tools and techniques Par Ian H. Witten,Eibe Frank