Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Autonomous Systems
0 likes151 views
The document discusses the integration of confidence monitoring and composition in autonomous systems to ensure safety in dynamic environments. It outlines the challenges in bridging design-time guarantees with run-time monitors and presents a structured approach involving formal verification, assumption effect analysis, monitoring, and composition of confidence measures. Future research directions are identified, focusing on dependencies between assumptions and automated functions for confidence composition.
Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Autonomous Systems
- 1. Confidence Monitoring and Composition for Dynamic Assurance of Learning-Enabled Autonomous Systems Ivan Ruchkin, Matthew Cleaveland, Oleg Sokolsky, Insup Lee University of Pennsylvania Klaus Havelund Festschrift October 24, 2021
- 2. Confidence in assurance Req: no obstacle collisions Req: no pipeline loss Safety verification Detection confidence Detection confidence Dynamics confidence How to combine confidences?
- 3. ● Goal: − Compute confidence in the guarantees of safety reqs − Given confidences from run-time monitors ● Challenge: − Design-time guarantees ←?→ run-time monitors Confidence in assurance
- 4. design-time phase run-time phase requirements no collisions no pipe loss environment system model dynamics control perception planning sensing
- 5. design-time phase run-time phase requirements no collisions no pipe loss verification environment system model dynamics control perception planning sensing
- 6. design-time phase run-time phase requirements no collisions no pipe loss verification system model dynamics control perception planning sensing ? environment
- 7. design-time phase run-time phase requirements no collisions no pipe loss verification “The obstacle is >10m away” “Vehicle follows the given dynamics equations” “No false-negative obstacle detections” “Reported distance is within 1m of true distance “ system model dynamics control perception planning sensing A1 A2 A3 A4 environment
- 8. design-time phase run-time phase requirements no collisions no pipe loss “The obstacle is >10m away” “Vehicle follows the given dynamics equations” “No false-negative obstacle detections” “Reported distance is within 1m of true distance “ system model dynamics control perception planning sensing A1 A2 A3 A4 environment
- 9. design-time phase run-time phase requirements no collisions no pipe loss “The obstacle is >10m away” “Vehicle follows the given dynamics equations” “No false-negative obstacle detections” “Reported distance is within 1m of true distance “ system model dynamics control perception planning sensing A1 A2 A3 A4 assumption effect analysis (A1 V A2) ∧ A3 ∧ A4 composition logic environment
- 10. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 obstacle assumption model M1 AM2 AM1 (A1 V A2) ∧ A3 ∧ A4 dynamics assumption model M2 AM4 AM3 requirements assumption models/monitors composition logic no collisions no pipe loss environment
- 11. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic no collisions no pipe loss M1 AM2 AM1 M2 AM4 AM3 obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment
- 12. probabilistic modeling system model design-time phase (vguar , cguar ) run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic (AM1 V AM2) ∧ AM3 ∧ AM4 assurance confidence monitor M1 AM2 AM1 M2 AM4 AM3 (v1 , c1 ) (v2 , c2 ) (v3 , c3 ) (v4 , c4 ) no collisions no pipe loss obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment
- 13. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic (AM1 V AM2) ∧ AM3 ∧ AM4 assurance confidence monitor M1 AM2 AM1 M2 AM4 AM3 (v1 , c1 ) (v2 , c2 ) (v3 , c3 ) (v4 , c4 ) no collisions no pipe loss obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment (vguar , cguar )
- 14. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic (AM1 V AM2) ∧ AM3 ∧ AM4 assurance confidence monitor M1 AM2 AM1 M2 AM4 AM3 (v1 , c1 ) (v2 , c2 ) (v3 , c3 ) (v4 , c4 ) no collisions no pipe loss obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment (vguar , cguar )
- 16. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” A2: “No false-negative obstacle detections” A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”
- 17. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Yes No A2: “No false-negative obstacle detections” A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”
- 18. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Yes No A2: “No false-negative obstacle detections” No Yes A3: “The obstacle is >10m away” A4: “Vehicle follows the given dynamics equations”
- 19. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Yes No A2: “No false-negative obstacle detections” No Yes A3: “The obstacle is >10m away” Yes Yes A4: “Vehicle follows the given dynamics equations”
- 20. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Yes No A2: “No false-negative obstacle detections” No Yes A3: “The obstacle is >10m away” Yes Yes A4: “Vehicle follows the given dynamics equations” Yes Yes
- 21. Assumption effect analysis Is this assumption required for R1: no collisions? Mode1: obstacle detected Mode2: obstacle not detected A1: “Reported distance is within 1m of true distance” Yes No A2: “No false-negative obstacle detections” No Yes A3: “The obstacle is >10m away” Yes Yes A4: “Vehicle follows the given dynamics equations” Yes Yes Composition logic: (Mode1 → A1 ∧ A3 ∧ A4) ∧ (Mode2 → A2 ∧ A3 ∧ A4)
- 22. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic (AM1 V AM2) ∧ AM3 ∧ AM4 assurance confidence monitor M1 AM2 AM1 M2 AM4 AM3 (v1 , c1 ) (v2 , c2 ) (v3 , c3 ) (v4 , c4 ) no collisions no pipe loss obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment (vguar , cguar )
- 23. ● Random variables: − Reported distance (RD) − True distance (TD) ● An assumption is an assertion over variables: − A1: | RD – TD | ≤ 1m (bounded error) − A2: RD = ∞ → TD = ∞ (no false negatives) ● Goal: compute queries over assns given observations − P( f(A1, A2) | RD ), where f is a given Boolean function Probabilistic modeling of assumptions
- 26. Assumption monitoring P(A1) = 0.78 P(A2) = 0.97 P(A1 V A2) = 0.98
- 27. probabilistic modeling system model design-time phase run-time phase dynamics control perception planning sensing A1 A2 assumption effect analysis A3 A4 (A1 V A2) ∧ A3 ∧ A4 requirements composition logic (AM1 V AM2) ∧ AM3 ∧ AM4 assurance confidence monitor M1 AM2 AM1 M2 AM4 AM3 (v1 , c1 ) (v2 , c2 ) (v3 , c3 ) (v4 , c4 ) no collisions no pipe loss obstacle assumption model M1 AM2 AM1 dynamics assumption model M2 AM4 AM3 assumption models/monitors environment (vguar , cguar )
- 28. Fundamental Research - Contract No. FA8750-18-C-0090 A1 A2 A1 ∧ A2 C(AM1, AM2) AM1 AM2 Safety Safe when satisfied, otherwise likely unsafe Calibrated, conservative confidence Conditional independence Calibrated, conservative confidence Options: - AM1*AM2 - w1*AM1 + w2*AM2 - AM1 + AM2 - 1 - LogReg(AM1, AM2) - Bayes(AM1, AM2) Confidence composition theory Calibrated, conservative confidence
- 29. Demo 1: guarantees hold, confidence high
- 30. Demo 2: guarantees fail, confidence drops
- 31. Future research directions ● Dependencies between assumptions and monitors ● Automated selection of composition functions ● Non-exhaustive design-time assurance: testing, simulation ● Recovery and adaptation based on composed confidence ● Second-order assumptions (of our assumption monitors)
- 32. Summary ● Monitoring confidence in assurance requires closing a gap: − Design-time guarantees ←?→ run-time monitors ● We address this problem in four steps: − Formal verification − Assumption effect analysis − Assumption monitoring − Composition of monitors