NeuroStrata: Harnessing Neuro-Symbolic Paradigms for Improved Testability and Verifiability of Autonomous CPS
Download as PPTX, PDF0 likes76 views
Presented by Xi (James) Zheng at the International Conference on the Foundations of Software Engineering (Ideas, Visions and Reflections Track) on June 24, 2025 in Trondheim, Norway. Abstract: Autonomous cyber-physical systems (CPSs) leverage AI for perception, planning, and control but face trust and safety certification challenges due to inherent uncertainties. The neurosymbolic paradigm replaces stochastic layers with interpretable symbolic AI, enabling determinism. While promising, challenges like multisensor fusion, adaptability, and verification remain. This paper introduces NeuroStrata, a neurosymbolic framework to enhance the testing and verification of autonomous CPS. We outline its key components, present early results, and detail future plans. Arxiv: https://arxiv.org/abs/2502.12267
![Early Results - Symbolic constraints for neural adaptation
7
• LLM extracts Linear Temporal Logic (LTL)
specifications from scene captions.
• Neural network generates Spatio-Temporal Scene
Graph (STSG) from video frames.
• Conformance checking compares STSG with LTL to
measure semantic alignment.
• Alignment score (probability of alignment) which is
approximated by underlying Scallop (alignment
checker)
• Violations are backpropagated as loss to improve
STSG generation
[1] J. Huang et al. “LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with
Weak Supervision
“ https://arxiv.org/abs/2304.07647](https://image.slidesharecdn.com/fseverificationvalidation11xizhengneurostrata-250628204711-e15b90a1/85/NeuroStrata-Harnessing-Neuro-Symbolic-Paradigms-for-Improved-Testability-and-Verifiability-of-Autonomous-CPS-7-320.jpg)
![Final step: Industry adaptation early effort-> Closed-loop UAV autolanding system adopted
and deployed by Australian drone delivery startup Skyy Network — ARC LP’22 [1], DSN’25 [2].
[1] "Robust and Scalable Autonomous Landing for Drones", CI, ARC Linkage Project grant LP210100337. Awarded
$459,593. 2022-2025.
[2] Sebastian Schroder, Yao Deng, Richard Han, Xi Zheng 。 Towards Robust Autonomous Landing Systems:
Iterative Solutions and Key Lessons Learned. IEEE/IFIP International Conference on Dependable Systems and
Networks, 2025, Naples, Italy](https://image.slidesharecdn.com/fseverificationvalidation11xizhengneurostrata-250628204711-e15b90a1/85/NeuroStrata-Harnessing-Neuro-Symbolic-Paradigms-for-Improved-Testability-and-Verifiability-of-Autonomous-CPS-10-320.jpg)
NeuroStrata: Harnessing Neuro-Symbolic Paradigms for Improved Testability and Verifiability of Autonomous CPS
- 1. NeuroStrata: Harnessing Neurosymbolic Paradigms for Improved Design, Testability, and Verifiability of Autonomous CPS Xi Zheng Ziyang Li Ivan Ruchkin Ruzica Piskac Miroslav Pajic
- 2. 2022 @ Macquarie University 2 Motivation & Context https://yachtharbour.com/news/what-you-need-to-know-about-superyacht-helipads-3229?src=news_view_page_bar "Robust and Scalable Autonomous Landing for Drones", CI, ARC Linkage Project grant LP210100337. Awarded $459,593. 2022-2025. • Autonomous CPSs (cars, drones, surgeries) increasingly rely on ML. • The uncertainty and black-box nature of ML increase system risk. • Critical needs in industry: Trustworthy, Interpretable, and Testable AI-based systems. https://kr-asia.com/three-dead-in-su7-crash-fatal-accident- renews-scrutiny-of-smart-driving-safety Tesla on autopilot crashes into overturned truck
- 3. 2022 @ Macquarie University 3 Problem Statement • Traditional ML (softmax, regression) lacks determinism and explainability. • Current formal methods do not scale or guarantee correctness in complex industry- grade CPS. • Formal verification of learning-enabled components remains input-output bounded robustness and simplified inputs/properties (e.g., left right collision avoidance) while learning-enabled systems are much more complex Yao Deng et al. 2022. Scenario-based test reduction and prioritization for multimodule autonomous driving systems. In FSE. 82–93. https://doi.org/10.1145/3540250.3549152
- 4. Limitations of Existing Approaches • Neural Network Verifiers: NNV, Sherlock, Reluplex, Alpha-Beta-CROWN Input-output properties only; lack full system guarantees • System-level Testing: Fuzzing, robustness analysis Broader coverage but rely on probabilistic reasoning → weaken formal guarantees Neurosymbolic Methods • Differentiable Logic Programming: TensorLog, DeepProbLog Deterministic, but complex mappings & limited multimodality • Program Induction: DreamCoder, Neuro-symbolic synthesis DSL rigidity limits adaptability for sensor fusion and reasoning Emerging Gaps • Lack of system-level formal guarantees • Limited integration across perception, planning, control • Runtime adaptability remains challenging
- 5. Proposed Solution - NeuroStrata 5 Hierarchical Architecture • High-Level (Symbolic): • Scene Graphs, Mission Planners • Program Induction (DreamCoder-Style) • Mid-Level (Neurosymbolic): • Semantic Maps, Local Planners • Symbolic constraints + neural adaptation • Low-Level (Neurosymbolic): • Sensor Fusion, Actuation Control • Impose Operational and Physical constraints + neural adaptation Core Idea • Combine neural adaptability with symbolic reasoning • Enable formal specifications across perception, planning, and control
- 6. Proposed Solution - NeuroStrata 6 Two-Phase Process • Top-Down Synthesis: • Generate symbolic & neurosymbolic modules from specifications • Bottom-Up Adaptation: • Runtime program induction to refine symbolic programs based on neural outputs Key Features • Specification mining via distillation + LLMs • Adaptive runtime behavior with formal guarantees • Integrated design-time synthesis & runtime verification
- 7. Early Results - Symbolic constraints for neural adaptation 7 • LLM extracts Linear Temporal Logic (LTL) specifications from scene captions. • Neural network generates Spatio-Temporal Scene Graph (STSG) from video frames. • Conformance checking compares STSG with LTL to measure semantic alignment. • Alignment score (probability of alignment) which is approximated by underlying Scallop (alignment checker) • Violations are backpropagated as loss to improve STSG generation [1] J. Huang et al. “LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision “ https://arxiv.org/abs/2304.07647
- 8. 2025 @ Macquarie University 8 Early Results – Semi-Automatically synthesized Landing Site Selection Module from Semantic Map via LLM & Human-in-the-loop for Drone auto-land system Fig.1 Our NeuroSymbolic approach compared with SOTA baselines Fig.2 Reasoning Capacities of Our NeuroSymbolic approach
- 9. Overall Plan 9 Step I — Dataset Generation • Model-based scenario synthesis with LLMs • Generate diverse, edge-case-rich, physically grounded datasets Step II — DSL Co-Design • Collaborate with domain experts • Express hierarchical specs across sensing, planning, control Step III — Specification Mining • Neurosymbolic distillation + LLM queries • Hybrid symbolic-neural techniques with active learning to refine mined specs Step IV — Design-Time Synthesis & Verification • Synthesize neurosymbolic modules from DSL specs • Compositional and abstraction-based verification for scalability Step V — Runtime Adaptation & Validation • Differentiable program induction for online symbolic updates • Low-overhead runtime monitoring for real-time validation
- 10. Final step: Industry adaptation early effort-> Closed-loop UAV autolanding system adopted and deployed by Australian drone delivery startup Skyy Network — ARC LP’22 [1], DSN’25 [2]. [1] "Robust and Scalable Autonomous Landing for Drones", CI, ARC Linkage Project grant LP210100337. Awarded $459,593. 2022-2025. [2] Sebastian Schroder, Yao Deng, Richard Han, Xi Zheng 。 Towards Robust Autonomous Landing Systems: Iterative Solutions and Key Lessons Learned. IEEE/IFIP International Conference on Dependable Systems and Networks, 2025, Naples, Italy
- 11. Q&A
Editor's Notes
- #5: First, I’ll introduce the background of autonomous driving system testing. The figure shows the architecture and data flow of a industry-level multi-module ADS. The ADS has many modules to implement different functionalities and finally output the control commands. Data flow among modules is based on the publish-subscribe mechanism. A module sends its output as a message into its corresponding publish channel, and another module will subscribe messages from the channel to get the needed input.
- #6: First, I’ll introduce the background of autonomous driving system testing. The figure shows the architecture and data flow of a industry-level multi-module ADS. The ADS has many modules to implement different functionalities and finally output the control commands. Data flow among modules is based on the publish-subscribe mechanism. A module sends its output as a message into its corresponding publish channel, and another module will subscribe messages from the channel to get the needed input.
- #7: First, I’ll introduce the background of autonomous driving system testing. The figure shows the architecture and data flow of a industry-level multi-module ADS. The ADS has many modules to implement different functionalities and finally output the control commands. Data flow among modules is based on the publish-subscribe mechanism. A module sends its output as a message into its corresponding publish channel, and another module will subscribe messages from the channel to get the needed input.
- #9: First, I’ll introduce the background of autonomous driving system testing. The figure shows the architecture and data flow of a industry-level multi-module ADS. The ADS has many modules to implement different functionalities and finally output the control commands. Data flow among modules is based on the publish-subscribe mechanism. A module sends its output as a message into its corresponding publish channel, and another module will subscribe messages from the channel to get the needed input.