Test Amplification in Python — An Industrial Experience Report
0 likes26 views
The document details a guest lecture on test amplification in Python, presented by Prof. Serge Demeyer, emphasizing the importance of software testing in detecting errors. It discusses the implementation of test amplification techniques, such as mutation coverage and examples of how to strengthen test cases, resulting in improved mutation scores and code coverage. Key findings include the success of the Amplifier tool in enhancing test validation in various software projects.
![Guest Lecture - Mälardalens Universitet © Serge Demeyer
Example - testDeposit_amplified (1/2)
6
1 def testDeposit_amplified (self) :
2 self.b.set_owner(’Iwena Kroka’)
3 self.b.deposit(10)
4 self.assertEqual(self.b.
5 get_transactions(), [10])
6 self.assertFalse(self.b.is_empty () )
7 self.assertEqual(self.b.owner, ’Iwena Kroka’)
8 self.assertEqual(self.b.get_balance(), 10)
…
Assertion
Am
plification](https://image.slidesharecdn.com/vasterastestamplification-221007112233-35d08f36/85/Test-Amplification-in-Python-An-Industrial-Experience-Report-6-320.jpg)
![Guest Lecture - Mälardalens Universitet © Serge Demeyer
Example - testDeposit_amplified (2/2)
7
1 def testDeposit_amplified (self) :
2 self.b.set_owner(’Iwena Kroka’)
3 self.b.deposit(10)
4 self.assertEqual(self.b.
5 get_transactions(), [10])
6 self.assertFalse(self.b.is_empty () )
7 self.assertEqual(self.b.owner, ’Iwena Kroka’)
8 self.assertEqual(self.b.get_balance(), 10)
9 with self.assertRaises(Exception):
10 self.b.deposit(−56313)
11 self.b.deposit(100)
12 self.b.set_owner(’Guido van Rossum’)
13 self.assertEqual(self.b.
14 get_transactions(), [10])
…
Input Am
plification](https://image.slidesharecdn.com/vasterastestamplification-221007112233-35d08f36/85/Test-Amplification-in-Python-An-Industrial-Experience-Report-7-320.jpg)
Test Amplification in Python — An Industrial Experience Report
- 1. Test Amplification in Python An Industrial Experience Report Prof. Serge Demeyer Guest lecture, Mälardalens Universitet, Västerås, Sweden October2022
- 2. Guest Lecture - Mälardalens Universitet © Serge Demeyer Testing 2 Program Under Test Input Expected output Software Testing is the process of executing a program or system with the intent of finding errors. (Myers, Glenford J., The art of software testing. Wiley, 1979
- 3. Guest Lecture - Mälardalens Universitet © Serge Demeyer Coverage 3 Program Under Test Input Expected output (mutation) coverage
- 4. Guest Lecture - Mälardalens Universitet © Serge Demeyer Test Amplification 4 Program Under Test Input Expected output (mutation) coverage +coverage Extra Input Extra Input +Extra Input +Extra output
- 5. Guest Lecture - Mälardalens Universitet © Serge Demeyer Example - testDeposit 5 1 def testDeposit (self) : 2 self.b.set_owner(’Iwena Kroka’) 3 self.b.deposit(10) 4 self.assertEqual(self.b.get_balance(), 10) 5 self.b.deposit(100) 6 self.b.deposit(100) 7 self.assertEqual(self.b.get_balance() , 210) I n p u t Expected output
- 6. Guest Lecture - Mälardalens Universitet © Serge Demeyer Example - testDeposit_amplified (1/2) 6 1 def testDeposit_amplified (self) : 2 self.b.set_owner(’Iwena Kroka’) 3 self.b.deposit(10) 4 self.assertEqual(self.b. 5 get_transactions(), [10]) 6 self.assertFalse(self.b.is_empty () ) 7 self.assertEqual(self.b.owner, ’Iwena Kroka’) 8 self.assertEqual(self.b.get_balance(), 10) … Assertion Am plification
- 7. Guest Lecture - Mälardalens Universitet © Serge Demeyer Example - testDeposit_amplified (2/2) 7 1 def testDeposit_amplified (self) : 2 self.b.set_owner(’Iwena Kroka’) 3 self.b.deposit(10) 4 self.assertEqual(self.b. 5 get_transactions(), [10]) 6 self.assertFalse(self.b.is_empty () ) 7 self.assertEqual(self.b.owner, ’Iwena Kroka’) 8 self.assertEqual(self.b.get_balance(), 10) 9 with self.assertRaises(Exception): 10 self.b.deposit(−56313) 11 self.b.deposit(100) 12 self.b.set_owner(’Guido van Rossum’) 13 self.assertEqual(self.b. 14 get_transactions(), [10]) … Input Am plification
- 8. Guest Lecture - Mälardalens Universitet © Serge Demeyer AmPyfier Engineering 8 AmPyfier: Test amplification in Python Journal of Software Evolution and Process 2022 https://doi.org/10.1002/smr.2490
- 9. Guest Lecture - Mälardalens Universitet © Serge Demeyer AmPyfier: Validation 9 Strengthened 37 out of 54 test classes
- 10. Guest Lecture - Mälardalens Universitet © Serge Demeyer SmallAmp: Validation 10 Strengthened 86 out of 156 test classes
- 11. 11 pull requests 8 merged 3 pending
- 15. Guest Lecture - Mälardalens Universitet © Serge Demeyer Python Project 15 20% 80% Project Test Suite • Start-up established in 2020 • Fast-growing - more customers - more investors - Single variant • AI engine is vital (Unit) Tests
- 16. Guest Lecture - Mälardalens Universitet © Serge Demeyer Data Driven Test Amplification 16 Amplification operators must manipulate .csv files import ampyfier @ampyfier.amplify_csv ( "CSVPATH" ) def test_something(amp_file="something.csv"): ...
- 17. 360 minutes
- 18. Guest Lecture - Mälardalens Universitet © Serge Demeyer Experimental Set-Up 18 Daily Ampyfier executes tests on main branch Weekly Ampyfier generates new tests
- 19. Guest Lecture - Mälardalens Universitet © Serge Demeyer Initial Amplification 19 Improvement for 13/25 files • Mutation Score: 20.29% • Code Coverage: 41.39%
- 20. Guest Lecture - Mälardalens Universitet © Serge Demeyer Continuous Amplification 20 Improvement for all • Mutation score: 7/8 ≈ 87% • Code coverage: 5/8 ≈ 63% • Both: 4/8 ≈ 50%