MUT4SLX: Extensions for Mutation Testing of Stateflow Models
1 like50 views
The document discusses mut4slx, a mutation testing tool for Simulink and Stateflow models, created by Prof. Serge Demeyer and collaborators. It provides an overview of the tool's features, including its capability to automate mutation testing through various operators and execute automatic tests with Signal Builder. Additionally, validation results from four small models are presented, demonstrating the effectiveness of the mutation testing process.
![© Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0.
Unit Testing: Streams of Values with Tolerated Deviation
10
public static void assertArrayEquals(String message,
float[] expected,
float[] actuals,
float delta)](https://image.slidesharecdn.com/vst2024presentationmut4slxfinal-240312154740-c9f9e44c/85/MUT4SLX-Extensions-for-Mutation-Testing-of-Stateflow-Models-10-320.jpg)
MUT4SLX: Extensions for Mutation Testing of Stateflow Models
- 1. MUT4SLX Extensions for Mutation Testing of Stateflow Models Speaker — Prof. Serge Demeyer Authors — Sien Nuyens, Halil Ibrahim Ceylan, Onur Kilincceker, Mutlu Beyazıt, Serge Demeyer
- 2. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. DevOps — Continuous Testing 2
- 3. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Build Pipeline 3 version control build developer test deploy (internal) automated system tests deploy (production) measure & validate ` ` merge conflicts compiler warnings regression faults deployment warnings unmet requirements deployment warnings key performance indicator M u t a t i o n T e s t i n g
- 4. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Testing The Quality Assurance 4 © Brussels Airlines
- 5. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Inject Synthetic Items 5 © "The Good, the Bad and the Ugly: Evaluating Convolutional Neural Networks for Prohibited Item Detection Using Real and Synthetically Composite X-ray Imagery”. Neelanjan Bhowmik, Qian Wang, Yona Falinie A. Gaus, Marcin Szarek, Toby P. Breckon
- 6. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. 6 Mostly textual programming languages (C++, C, C#, Java, Python, …)
- 7. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Industry 4.0: Cyber-Physical Systems 7
- 8. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. MatLab Ecosystem Simulink • sensors and actuators (continuous signals) • embedded (control) software StateFlow • reactive systems • combinatorial & sequential decision logic 8
- 9. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Microwave (StateFlow) 9 input = stream of values • start boolean; • door_closed boolean; • clear boolean; • steps_to_cook int output = stream of values • mode int
- 10. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Unit Testing: Streams of Values with Tolerated Deviation 10 public static void assertArrayEquals(String message, float[] expected, float[] actuals, float delta)
- 11. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Test Design with Signal Builder 11 Assert Compare expected against actual signal
- 12. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Assert with SignalBuilder: Compare Signals 12 Assert Compare expected against actual signal
- 13. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. MUT4SLX: Mutation Testing for Simulink … 13
- 14. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. MUT4SLX: Mutation Testing for Simulink and StateFlow 14 New! VST2024
- 15. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. MUT4SLX Overview 15 Model Working Directory Target System Mutation Operators Set Configuration Start Matlab Open Working Directory Open Model Select Mutation Operators Find Specific Block Type for Mutation Generate Mutants Load JSON File Save Mutants on CSV File Pre-run Test Apply Mutation on Model Run-Test Check Test Results Prepare Report Detect Test Engine MUT4SLX Yes No Untested Mutant in CSV? Save Report in HTML format MUT4SLX Fast mutant injection by manipulating via the API Automatic test execution via Signal builder and Simulink test.
- 16. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. 16
- 17. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Mutation Operators: Industrial Collaboration 17 # Name Description 1 ROR Relational Operator Replacement 2 LOR Logical Operator Replacement 3 ASR Arithmetic Sign Replacement 4 MMR Min-Max Replacement 5 ICR If Condition Replacement 6 TOR Trigonometric Operator Replacement 7 MOR Math Operator Replacement 8 PMR Product Multiplication Replacement 9 POR Product Operator Replacement 10 FIR For Index Replacement 11 FLR For Limit Replacement 12 UDO Unit Delay Operation 13 STR Switch Threshold Replacement 14 SCR Switch Criteria Replacement 15 CR Constant Replacement # Name Description 1 ROM Relational Operator Mutation 2 LOM Logical Operator Mutation 3 BOM Bitwise Operator Mutation 4 MOM Math Operator Mutation 5 NOM Non-bitwise Operator Mutation 7 CCM Control Chart Mutation 8 CDR Control chart Duration Mutation 9 TOM Transition Order Mutation New!
- 18. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. Validation: 4 small models 18 System # mutants # killed mutants mutation score mutant generation time mutant execution time Cruise Control 63 33 52.38% 1.4 sec 11.79 min Microwave 30 21 70% 0.7 sec 7.68 sec Aircraft Elevator 2243 - - 66.1 sec - Automatic Transmission 53 - - 1.35 sec - System # subsystems Total Block Count Number of States Number of Transition Number of tests Cruise Control 1 9 7 23 15 Microwave 3 17 4 11 1 Aircraft Elevator 37 825 29 104 0 Automatic Transmission 17 65 9 14 0 It works … … fast!
- 19. © Copyright Serge Demeyer 2024. Licensed under CC BY_NC 4.0. 19 https://github.com/haliliceylan/MUT4SLX