Neuro-Symbolic Bridge: From Perception to Estimation & Control
0 likes24 views
A poster presented by Radoslav Ivanov and Ivan Ruchkin at the NSF CPS Principal Investigators meeting on March 13, 2025.
Neuro-Symbolic Bridge: From Perception to Estimation & Control
- 1. Neuro-Symbolic Bridge: From Perception to Estimation & Control PIs: Radoslav Ivanov (RPI), Ivan Ruchkin (UF) Students: Thomas Waite, Shuhang Tan (RPI), Yuang Geng, Trevor Turnquist (UF) Rensselaer Polytechnic Institute, University of Florida 1. State-Dependent Conformal Perception Bounds for Neuro-Symbolic Verification of Autonomous Systems Problem: CPS rely on neural networks (NNs) for perception and symbolic techniques for estimation/control. A fundamental mismatch exists between the uncertainties in NN outputs and the assumptions of symbolic tasks, compromising system safety and performance. Key idea: A neuro-symbolic calibration and training to bridge the gap between NN perception and downstream symbolic tasks: ● Be robust to sensor faults and perception errors. ● Align NN uncertainties with symbolic task assumptions (e.g., bounded or Gaussian noise). ● Ensure temporal consistency of calibrated outputs with system dynamics. Broader Impacts: ● Mentoring: PI Ivanov is advising a high school student. PI Ruchkin is mentoring several undergraduates. ● Workshop on neuro-symbolic CPS: PI Ruchkin is co-chairing a CAV’25 workshop on neuro-symbolic CPS https://www.tacps.org Approach Overview: Project info: CPS Small, $499k 06/2024-05/2027 Awards ID#: 2403615, 2403616 - Pre-trained NN perception processes raw sensor data. - Thrust I: Extrinsic Neuro-Symbolic Calibration (post-training) - Thrust II: Intrinsic Neuro-Symbolic Calibration (training & calibration) - Symbolic estimation: calibrated NN outputs for improved safety and performance. Problem: given system dynamics and a dataset of IID trajectories, construct a sequence of reachable sets such that: ● Intuition: perception errors vary across state space; can we design a state-dependent conformal prediction method? Solution: ● Split state space into regions to minimize conformal error ○ Defined weighted loss of accumulated error ● Synthesize state regions with gradient-free optimization (genetic search and simulated annealing) 2. Verifiable Neural Approximation of Vision Control - Approximate a high-dimensional controller (HDC) with multiple low-dimensional controllers (LDCs) - Three statistical discrepancies between HDC and LDCs TEA Lab Y. Geng, J. Baldauf, S. Dutta, C. Huang, I. Ruchkin. Bridging Dimensions: Confident Reachability for High-Dimensional Controllers. In Proc. of FM’24. Results: T. Waite, Y. Geng, T. Turnquist, I. Ruchkin, R. Ivanov. State-Dependent Conformal Perception Bounds for Neuro-Symbolic Verification of Autonomous Systems. ArXiv preprint, in submission. 3. Analyzing Neural Network Robustness Using Graph Curvature - A fresh look at neural network (NN) robustness from a graph theory point of view - Use Ricci Curvature to identify bottleneck NN edges that contribute to low robustness S. Tan, J. Sia, P. Bogdan, R. Ivanov. Analyzing neural network robustness using graph curvature. In 2024 International Conference on Assured Autonomy (ICAA) (pp. 110-113). IEEE.