Improving software quality and devop automation with STAMP, OW2con'16, Paris.
0 likes386 views
The document outlines the objectives of the STAMP project, which aims to automatically enhance the quality of test assets in DevOps by detecting regression bugs pre-production. Key techniques discussed include continuous test amplification and observation space amplification, leveraging transformations of existing test cases to improve software testing efficiency. The project, involving nine EU partners, started on December 1, 2016, and focuses on integrating advanced testing methodologies within microservice architectures.
Improving software quality and devop automation with STAMP, OW2con'16, Paris.
- 1. STAMP Software testing amplification to increase quality in DevOps 1
- 2. 2 DevOps
- 3. 3 DevOps Test assets are available BUT They are manually developed and their quality is not sufficient
- 4. 4 DevOps The main objective of STAMP is to automatically transform existing test assets in order to detect regression bugs before production and drive down the cost of software testing.
- 9. Continuous test amplification 9 behav. diff. perf. diff.
- 10. Continuous test amplification 10 behav. diff. perf. diff.
- 11. Continuous test amplification 11 behav. diff. perf. diff.
- 12. Unit test cases amplification 12 Program P Input space Output space state
- 13. Unit test cases amplification 13 Program P Input space Output space state
- 14. DSpot: amplification of JUnit test suites 14 original test suite and program under test update in the program under test P DSpot TS P' TS++
- 15. DSpot •First experiments with amplification •Automatic transformation of Junit test cases 15 P TS P' TS+ search-based input space amplification observation space amplification run and look for differences TS++
- 16. Search-based input space amplification • Mutate existing test cases • transform literal values • replace literal values by others found in the scope • add new invocations to methods not well covered • create new objects • Search-based • ‘stack’ transformations iteratively • select the ones that have a good fitness (e.g. cover new branches) 16
- 17. Observation space amplification • Remove all assertions from original test cases • keep invocations that can be inside the assertion • Add new observation points • run all test cases on original program • record all values that can be observed (getters, is_*, etc.) • Filter new observation points • several runs of all test cases with observation points on the original program • discard all points that vary (e.g., OS-dependent values, IO, etc.) 17
- 18. Example 18 public void testSearch() { final ArrayStack<E> stack = makeObject(); stack.push(((E)("First Item"))); stack.push(((E)("Second Item"))); assertEquals("Top item is 'Second Item'", 1, stack.search("Second Item")); assertEquals("Next Item is 'First Item'", 2, stack.search("First Item")); assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item"));} public void testSearch_cf5() { final ArrayStack<E> stack = makeObject(); stack.push(((E)("First Item"))); stack.push(((E)("Second Item"))); E o_5 = stack.peek(); assertFalse(((java.lang.String)o_5).isEmpty()); E o_6 = stack.pop(); assertFalse(((java.lang.String)o_6).isEmpty()); Object o_7_0 = stack.search("Second Item"); Object o_9_0 = stack.search("First Item"); assertEquals("Cannot find 'Missing Item'", -1, stack.search("Missing Item")); assertEquals(((java.lang.String)o_5).length(), 11); assertEquals(((java.lang.String)o_6).length(), 11);} new calls and asserts extracted from asserts new asserts
- 19. Methodology 19 Unit test ampli Config. test ampli Online test ampli
- 20. Methodology 20 Micro service architecture Unit test ampli Config. test ampli Online test ampli
- 21. Methodology 21 Micro service architecture Unit test ampli Config. test ampli Online test ampli Information management Software quality Cloud computing E-Health Smart cities
- 22. STAMP •3-year project •9 EU partners •Full OSS •Starts 01/12/16 22