Controller Software Verification Using AVM Meta and HybridSAL
4 likes864 views
The document discusses using the AVM Meta tool suite and HybridSAL to formally verify controller software for cyber-physical systems. It describes how controllers modeled in Simulink/Stateflow can be integrated with physical models in Modelica. Design space exploration is used to simulate different controller alternatives. Formal verification of properties specified in temporal logic is then used to detect errors in the candidate controller. The workflow involves translating controllers to a cyber language, generating simulation code, and visualizing verification results. Counterexamples can provide insight to refine the controller model or property specification.
 )
• []( [](srs==3 && dgs==7) => <>(gear==4) )
2. If dgs==6(low), then eventually gear <= 1
• []( [](dgs==6) => <>(gear <= 1) )
• []( [](dgs==6) => <>(gear <= 2) )](https://image.slidesharecdn.com/verificationcps-140702150840-phpapp02/85/Controller-Software-Verification-Using-AVM-Meta-and-HybridSAL-28-320.jpg)
 ) Violated
• []( [](srs==3 && dgs==7) => <>(gear==4) ) Verified
2. If dgs==6(low), then eventually gear <= 1
• []( [](dgs==1) => <>(gear <= 1) ) Violated
• []( [](dgs==1) => <>(gear <= 2) ) Verified
3. The event gear_selected==4 is absent before
dgs is 7 (drive) Verified](https://image.slidesharecdn.com/verificationcps-140702150840-phpapp02/85/Controller-Software-Verification-Using-AVM-Meta-and-HybridSAL-35-320.jpg)
 => <>(gear <= 1) ) Violated](https://image.slidesharecdn.com/verificationcps-140702150840-phpapp02/85/Controller-Software-Verification-Using-AVM-Meta-and-HybridSAL-38-320.jpg)
 => <>(gear <= 2) ) Verified](https://image.slidesharecdn.com/verificationcps-140702150840-phpapp02/85/Controller-Software-Verification-Using-AVM-Meta-and-HybridSAL-40-320.jpg)
 => <>(gear <= 1) ) Verified](https://image.slidesharecdn.com/verificationcps-140702150840-phpapp02/85/Controller-Software-Verification-Using-AVM-Meta-and-HybridSAL-45-320.jpg)
Controller Software Verification Using AVM Meta and HybridSAL
- 1. SRI International Verification of Cyber-Physical Controller Software Using the AVM Meta Tool Suite and HybridSAL Joseph Porter (jporter@isis.vanderbilt.edu), Ashish Tiwari (ashish.tiwari@sri.com), and Xenofon Koutsoukos (xenofon.koutsoukos@vanderbilt.edu) S5 Symposium June 2014
- 2. SRI International Controller Design Design space exploration with simulation can answer early questions about controller choices • Simulation over alternatives is much cheaper than building prototypes • Automated DSE helps answer questions about the evolving design Formal verification tools can detect hard-to-find errors • Testing may fail to detect (large scenario space) • Requires property specification • CyPhy enables easy property addition, model editing, and visualize verification results
- 3. SRI International OpenMETA-CyPhy Cyber Toolchain (DARPA AVM) Physical Components C2M2L Components Custom Components Cyber Components CyPhy System Design / Design Space Dynamics TB Verification TB Physical Components Authoring Cyber Components Authoring * Cyber Modeling Language Component Use Simulink/ Stateflow Auto Import Component Use Modelica Sim (.mo, .lib) CyPhy Modelica Composer Cyber Code Generator HybridSAL DAE XML Cyber XML Verification Condition Generation System Design Space Authoring Hybrid System Evaluation Dymola/ OpenModelica Modelica Simulation Formal Verification
- 4. SRI International Everything has software – cars, refrigerators, even chainsaws. How do we model controller software and combine it with the mechanical design? • Keep the design processes for the physical and cyber in sync. • Building and testing prototypes is expensive. Integrating controllers into mechanical designs is costly and time-consuming. Too many scenarios to test cyber and physical systems exhaustively. Testing will miss important cases. Some examples: • Toyota Acceleration problem • MARS polar lander • Ariane 5 Why Cyber Modeling?
- 5. SRI International AVM Control Design Capabilities How are META Cyber capabilities different from designing and simulating controllers using Modelica or Simulink alone? 1. Cyber controller component in Modelica is not a simulation model, but is the actual embedded code scheduled by a discrete-time periodic sampler. The simulation is much more realistic. 2. Integrated formal hybrid verification toolchain. 3. All of the benefits of using CyPhyML: • design space exploration • curated component library • integrated test bench models • parametric exploration • cloud-based simulation
- 6. SRI International Introduction We will show how the Cyber tools can integrate multiple transmission controllers modeled in Simulink/Stateflow with a Driveline modeled in Modelica. We will use DSE to assess the controller alternatives in simulation, and then verify the candidate controller. Scenario: If we have a cheaper fixed-point processor, will the controller performance be degraded?
- 7. SRI International Creating a CyPhy model The CyPhy model is the “integration” model. It contains: • The imported Modelica model (The System) • The interface to the Cyber model (The Controller) • A design describing how the two are connected • A test bench describing how to evaluate the design
- 8. SRI International Connecting the Cyber interface to the system model in GME All modern designs have software controllers. How do we incorporate them into a design?
- 9. SRI International CyPhyML Controller Composition Concepts Physical Component S V B Modelica Component P AVM Component Interface (Dynamics) • Connectors • Parameters Modelica Component Interface • Signals (causal) • Physical variables (acausal) • Buses (aggregate) • Parameters Modelica Components and Ports each have a Class string indicating the location of its type definition in the library. C2M2L_Ext.Interfaces.Context_Interfaces.Driver.Driver_Bus
- 10. SRI International CyPhyML Controller Composition Concepts Physical Component S V B Modelica Component P AVM Component Interface (Dynamics) • Connectors • Parameters Modelica Component Interface • Signals (causal) • Physical variables (acausal) • Buses (aggregate) • Parameters Modelica Components and Ports each have a Class string indicating the location of its type definition in the library: C2M2L_Ext.Interfaces.Context_Interfaces.Driver.Driver_Bus
- 11. SRI International CyPhyML Controller Composition Concepts Physical Component S V B Modelica Component P Modelica Components and Ports each have a Class string indicating the location of its type definition in the library. C2M2L_Ext.Interfaces.Context_Interfaces.Driver.Driver_Bus Component B In composition, the underlying port types must match exactly (by type string).
- 12. SRI International CyPhyML Controller Composition Concepts Physical Component S V B Modelica Component P Controller Component B S P Controller components must be causal: • Only connect to signals and bus ports containing signals. • Physical variable ports can not be connected to controller components. X Parameter ports are exposed in assemblies and test benches. Parameter values are propagated down into the components.
- 13. SRI International CyPhyML Controller Composition Concepts Physical Component S V B Modelica Component (refers to external model in Modelica component libraries)P Controller Component B S P X Cyber Component (refers to external Cyber model) S B P AVM and Cyber component model interfaces are identical. Simulink S S S Bus ports aggregate signals. Bus signals are identified by name. Only one component can produce (output) a bus signal. The others must consume it. P Cyber interface parameters refer directly to parameter objects deep in the Simulink hierarchy.
- 14. SRI International Translating controllers to the Cyber language MDL2MGA Translator Controllers in Simulink/Stateflow Controllers in GME Cyber language Shift controller Torque converter controller Torque reduction controller
- 15. SRI International Design Space Exploration DSE allows us to evaluate different controller variants, to compare performance.
- 16. SRI International CyberComposition Model (GME) ControllerModelsDynamicsModels Controller Details Modelica Interface CyPhyML Model (GME) Modelica Controller References Modelica Dynamic Components Modelica Dynamics References Modelica Design Simulation Dynamics Models Controller Libraries Legend Reference from CyPhyML to an external component model. Model created manually/semi- automatically. Automatic model translation step. 1 Generated Code 2 5 6b 6a Simulink/ Stateflow 9 Controller Components 4 Design Space 3 Test Bench7 8a Controller C Code “Glue” Code Modelica Interface 8b Controller Import Code Generation Physical Component Import System Modeling Generation & Simulation META-Cyber Simulation Workflow
- 17. SRI International META-Cyber Simulation Workflow 1. Create or obtain physical component models in Modelica. 2. Create a CyPhyML model using GME. 3. Import Modelica components and build the system design space in the CyPhy model. 4. Create your controllers in Simulink/Stateflow. 5. Create a Cyber model in META and import your controllers into this model. 6. Define the controller interface in the CyPhyML model using GME (6a) and link it to the Cyber model (6b). 7. Create a test bench for the design in CyPhyML. 8. From the test bench, run the Master Interpreter to generate Modelica simulation code (8a). The Modelica generator also invokes the code generator to create Modelica controller components which are integrated into the simulation (8b). 9. Run the simulation.
- 18. SRI International Tool Flow Test Bench model defines inputs & environment for system under test Invoke DSE to explore alternatives Designs Component Alternatives DESERT tool presents design choices Select designs to generate Specify component Library paths Generate the Modelica model Simulate the designs and compare
- 19. SRI International Generated Code Generated code is placed in Simulink subdirectory Generated code can be inspected by opening the generated Visual Studio solution file.
- 20. SRI International Alternative #1 Simulation results #1 Selected gear vs time (s) Engine rpm vs time (blue) Transmission output rpm vs time (red)
- 21. SRI International Selected gear vs time (s) Simulation results #2 Alternative #2 Engine rpm vs time (blue) Transmission output rpm vs time (red)
- 22. SRI International Controller Correctness Once we have identified our candidate controller, we can use formal verification to assess the controller logic.
- 23. SRI International Formal Verification Testing/simulation can fail to detect design errors that are manifested only under certain scenarios Formal verification can verify designs for all scenarios Formal verification complements simulation in improving confidence in system design
- 24. SRI International Formal Verification What is formal verification? Techniques for verifying the system that are based on symbolic algebra, rather than numerical simulations Achieves the equivalent of exhaustive testing
- 25. SRI International Verification Workflow Identical to the simulation workflowexcept: Temporal properties are attached to models Verification results, namely o status (pass, fail, or error) and explanation are displayed on the dashboard Controllers often include intricate logic that can be difficult to exhaustively test Apply formal verification to ShiftController here
- 26. SRI International Temporal Properties Properties capture the intent of the controller design ShiftController: Inputs: 1.driver_gear_select 2.shift_request_state 3.input_speed_TC 4.output_speed_TC Output: 1.gear_selected
- 27. SRI International Shift Controller Inputs-Output The input variable driver_gear_select takes values: reverse=1, park=2, swim=3, neutral=4, neutral_pivot=5, low=6, drive=7 The input variable shift_request_state takes values: down_shift=1, no_shift=2, up_shift=3 The output variable gear_selected takes values: 0, 1, 2, 3, 4 A correct controller should guarantee something about the output under certain assumptions on the inputs.
- 28. SRI International Temporal Properties as Specification Some desired properties for such a ShiftController: 1. If shift_request_state==3 (up_shift) and driver_gear_select==7 (drive), then eventually gear_selected==4 (fourth gear) • []( srs==3 && dgs==7 => <>(gear==4) ) • []( [](srs==3 && dgs==7) => <>(gear==4) ) 2. If dgs==6(low), then eventually gear <= 1 • []( [](dgs==6) => <>(gear <= 1) ) • []( [](dgs==6) => <>(gear <= 2) )
- 29. SRI International Pattern-based Property Specification All properties above have the same “Global Response’’ pattern Another useful pattern is “Absence before R’’ Property: “The output gear_selected does not take value 4 before driver_gear_select is 7 (drive)” P := gear_selected == 4 R := driver_gear_select == 7 Property: P is absent before R
- 30. SRI International Adding LTL Properties to SL/SF Models
- 31. SRI International Pattern-based Property Specification If driver_gear_select is 6(low), then eventually gear_selected <= 1 (first gear)
- 32. SRI International Viewing Properties in SL/SF Models Set of all properties = Specification of the component
- 33. SRI International The Verification TestBench After controller models have been annotated with desired temporal properties, they are translated into CyberComposition language, and verified SL/SF + Properties CyberComp XML Formal Verification Results Verification results can be viewed in the dashboard
- 34. SRI International Verification Results in the Dashboard
- 35. SRI International Understanding Verification Results 1. If shift_request_state==3 (up_shift) and driver_gear_select==7 (drive), then eventually gear_selected==4 • []( srs==3 && dgs==7 => <>(gear==4) ) Violated • []( [](srs==3 && dgs==7) => <>(gear==4) ) Verified 2. If dgs==6(low), then eventually gear <= 1 • []( [](dgs==1) => <>(gear <= 1) ) Violated • []( [](dgs==1) => <>(gear <= 2) ) Verified 3. The event gear_selected==4 is absent before dgs is 7 (drive) Verified
- 36. SRI International Property is False in the Model: Details
- 37. SRI International Simulating the Counter-Example
- 38. SRI International Visualizing the Counter-Example []( [](dgs==6) => <>(gear <= 1) ) Violated
- 39. SRI International Refining the Model or Property When a property is found to be false in the model, the user can view and analyze the counter- example, and based on that, go back to the controller design and fix either • The controller • Edit transition guards • Add/delete transitions • The property • Make the property weaker • Constrain the inputs of the controller
- 40. SRI International Refining the Property Since output variable, gear_selected, gets stuck at value 2, we can see if our controller satisfies a weaker specification. []( [](dgs==6) => <>(gear <= 2) ) Verified
- 41. SRI International Refining the Model Designer changes condition on the downshift transition out of Gear2 in SL/SF
- 42. SRI International Refining the Model The condition on the outgoing transition from state Gear2 to state lockoutD1 is chaged from: srs==1 && dgs!=6 && dgs!=1 && in_tc > out_tc ↓ ((srs==1 && dgs!=6 && dgs!=1 && in_tc > out_tc) || (dgs==6 || dgs==1)) Enable transition additionally when driver_gear_select is 6 (low)
- 43. SRI International Verification Results for the Updated Model
- 44. SRI International Property is True in the Model: Details
- 45. SRI International Old CounterExample on the Fixed Model []( [](dgs==6) => <>(gear <= 1) ) Verified
- 46. SRI International Conclusions Design space exploration with simulation can answer early questions about controller choices • Simulation over alternatives is much cheaper than building prototypes • Automated DSE helps answer questions about the evolving design Formal verification tools can detect hard-to-find errors • Testing may fail to detect (large scenario space) • Requires property specification • CyPhy enables easy property addition, model editing, and visualize verification results