Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation
0 likes257 views
The document presents an adaptive approach called Adynamosa for performance-aware automated test case generation, which aims to achieve code coverage while reducing resource demands. A study of 110 Java classes showed that the method maintained similar branch coverage and mutation scores compared to traditional methods, while achieving approximately 70% lower runtime and memory consumption. The findings highlight the potential for optimizing testing processes without compromising software quality.
![Coverage vs non-coverage criteria
maximize code coverage criteria
other desirable qualities
(readability[1], code quality,
execution time, memory usage)
codecoverage
non cov. criteria
[1] Daka et.al - Modeling readability to improve unit tests (ESEC/FSE 2015)](https://image.slidesharecdn.com/icse-tse-200701060638/85/Testing-with-Fewer-Resources-An-Adaptive-Approach-to-Performance-Aware-Test-Case-Generation-3-320.jpg)
![Case Study
110 Java classes
27 different projects
3 research questions
Can we achieve the same
branch coverage?
Can we achieve the same
mutation score?
Can we reduce test runtime and
heap memory consumption?
DynaMOSA[2]
[2] Panichella et.al - Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets](https://image.slidesharecdn.com/icse-tse-200701060638/85/Testing-with-Fewer-Resources-An-Adaptive-Approach-to-Performance-Aware-Test-Case-Generation-8-320.jpg)
Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation
- 1. Testing with Fewer Resources: An Adaptive Approach to Performance-Aware Test Case Generation Giovanni Grano, Christoph Laaber, Annibale Panichella, Sebastiano Panichella IEEE Transaction on Software Engineering JF @ ICSE 2020
- 2. testing activities high quality software expensive evolutionary search Automated Test Case Generation
- 3. Coverage vs non-coverage criteria maximize code coverage criteria other desirable qualities (readability[1], code quality, execution time, memory usage) codecoverage non cov. criteria [1] Daka et.al - Modeling readability to improve unit tests (ESEC/FSE 2015)
- 4. aDynaMOSA Execution time and heap memory consumption Harm the coveragePerformance measurement 2 main challenges
- 5. Performance Proxies proper measurement unfeasible 7 performance proxies performance score 1000 times
- 6. Algorithm algorithms population offsprings ranks crowding distance generation
- 7. Algorithm algorithms population offsprings ranks crowding distance performance score generation
- 8. Case Study 110 Java classes 27 different projects 3 research questions Can we achieve the same branch coverage? Can we achieve the same mutation score? Can we reduce test runtime and heap memory consumption? DynaMOSA[2] [2] Panichella et.al - Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets
- 9. RQ1 & RQ2 branch coverage mutation score 72 vs 72 79 out of 110 32 vs 33 85 out of 110
- 10. RQ1 & RQ2 branch coverage mutation score 72 vs 72 79 out of 110 32 vs 33 85 out of 110 same achieved code coverage and mutation score
- 11. RQ3 subjects no difference in coverage rigorous profiling runtime and memory consumption ~70% lower runtime and memory consumption
- 12. RQ3 subjects no difference in coverage rigorous profiling runtime and memory consumption lower resource demand with same coverage ~70% lower runtime and memory consumption
- 13. Conclusions aDynaMOSA - adaptive approach focus on tests resource demand different secondary objectives @giograno90 https://arxiv.org/abs/1907.08578