Fahroo - Dynamics and Control - Spring Review 2013
1 like816 views
The document outlines the portfolio of Dr. Fariba Fahroo, focusing on the development of mathematical theories and algorithms for control systems in uncertain environments. It covers various research areas including control theory, distributed multi-agent control, and human-machine interactions, with specific challenges like adversarial behaviors and achieving optimal coordination in communication-limited scenarios. Key achievements include advancements in multi-agent autonomy and task allocation methodologies, with applications in diverse fields such as quantum control and autonomous vehicle operations.
Fahroo - Dynamics and Control - Spring Review 2013
- 1. 1DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution14 February 2013 Integrity Service Excellence Dr. Fariba Fahroo Program Officer AFOSR/RTA Air Force Research Laboratory Dynamics & Control Date: 4 March 2013
- 2. 2DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution 2013 AFOSR SPRING REVIEW NAME: Fariba Fahroo BRIEF DESCRIPTION OF PORTFOLIO: Developing mathematical theory and algorithms based on the interplay of dynamical systems and control theories with the aim of developing innovative synergistic strategies for the design, analysis, and control of AF systems operating in uncertain, complex, and adversarial environments. LIST SUB-AREAS IN PORTFOLIO: • General Control Theory: optimal control, adaptive control, stochastic control, hybrid control • Distributed Multi-Agent Control: path planning, decision making, sensing, task allocation with adversarial and stochastic elements with incomplete information and communication constraints • Mixed Human-Machine Interface • Control of Distributed Parameter Systems • Emerging Applications: Quantum Control, Vision-based Control
- 3. 3DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Key Portfolio Research Challenges Key enabling research areas in the science of autonomy •Autonomous Dynamic Mission Planning : Hybrid System Formulation (mix of finite, discrete state (decision variables) and continuous Dynamics) • Human-Machine Interactions and operations: Stochastic Dynamic Programming, Stochastic Control • Incorporation of Uncertainty in Mission Environment, Model Parameters: Adaptive Control, Stochastic Control • Adversarial Behavior: Game theoretic Approaches, Stochastic game theory (mean-field theory) • Incomplete Information: incorporation of learning into games, task allocation, planning • Computational Issues: Computational Nonlinear Control Theory • Emergent Applications: Quantum Control, Vision Control
- 4. 4DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Distributed Control for Networked Systems • Objectives: To address fundamental problems in distributed control of networked systems with non-traditional constraints and adversarial action. • Challenges: A comprehensive research program that deals, within a multi-agents context, – with decentralization, – localization of objectives, – lossy communication and information exchange, – incompleteness of information, – adversarial interference, – multiplicity of objectives, – coordination through multi-level interaction. • Modeling Framework: deterministic, stochastic and hybrid structures, cast in discrete- as well as continuous-time settings, with a strong mathematical underpinning. • Multi-Disciplineary Research: Control theory, Information theory, Decision theory, Networks, Coding, Communication, Computing, and Game theory.
- 5. 5DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Multi-Agent Autonomy How (MIT) • Vision: Team of agents that share experiences and learn to act more intelligently in dynamic, uncertain, & contested environments – Learn/update models & planners/policies to improve mission performance • Approach: Intelligent Cooperative Control Architecture (iCCA) that builds/learns effectively from prior knowledge – Synergistic integration of planning and learning to improve performance – Bounds risk in learning process • News: – iCCA with adaptive modeling shown to out-perform individual methods – Developed new model learning techniques that greatly reduce sample complexity of iCCA – Performance improvement demonstrated in experimentally using quadrotors CBBA CBBA + Sarsa iCCA Sarsa Sarsa CBBA conservative CBBA Aggressive iCCA iCCA Adaptive Modeling
- 6. 6DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Distributed Task Allocation • Goal: Automate mission planning to improve performance of networked team of heterogeneous agents operating in a contested environment – Distributed/decentralized planning is required for some communication environments (e.g. slow, high cost, unreliable, ...). – Planning issues: convergence guarantees/time, mission performance • Approach: Consensus-Based Bundle Algorithm (CBBA) developed as multi- assignment task allocation algorithm – Recent results highlighted the performance limitations associated with the score functions that must be used to guarantee convergence • News: Developed bid-warping approach for CBBA – Expands the set of score functions that can be used – Can use cost functions that make the most sense – Provable bound on performance
- 7. 7DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Time-Critical Cooperative Missions for Multi-Agent Systems Naira Hovakimyan (UIUC) Time-critical applications for multiple vehicles with spatial constraints: • Sequential auto-landing (UAVs) • Coordinated reconnaissance missions • Simultaneous arrival at multiple locations How to: 1) Efficiently generate trajectories that satisfy desired relative timing constraints? 2) Enforce these constraints with desired performance levels in communication- limited or communication-deprived environments? Courtesy: NPS Courtesy: LARSyS, IST
- 8. 8DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Coordination in Communication-Limited or -Deprived Environments Connectivity of the communications network satisfies: • Graph connected in an integral sense, not piecewise in time Rate of convergence of the coordination dynamics proportional to the connectivity level: Low connectivity implies slow convergence Potential directions for research: • Use of onboard estimators: can improve learning of other agents’ dynamics… but leads to larger networks (characterized by smaller convergence rates). • Topology control is needed to decide which estimators to listen to and which not: ‘Edge snapping’ based on link quality and coordination error. How to modify coordination control algorithms to improve convergence rate?
- 9. 9DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Autonomous Optimal Pursuit David Casbeer, Krishna Kalyanam, Phil Chandler, AFRL/RQ Meir Pachter, AFIT, Swaroop Darbha, TAMU • Problem Formulation: Develop practical and autonomous decision-making algorithms under uncertainty • UGS monitors a road network, while a UAV(s) continually visits the UGS for status updates • When an UGS alerts a UAV of an intruder, the UAV must decide where to go to capture/image the intruder and deliver imagery to operator • Challenging Problem: Stochastic optimization with dual-control aspects including partial and delayed observations • Very few problems of this type have been addressed • Solution does not require complete information history (implementable) • Developed probabilistic guarantees for autonomous capture in stochastic setting • Transition: Flight test and demonstration
- 10. 10DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Flight Test (Transition) – Onboard computation and UAV-to-UGS communication have been designed, tested, and characterized • An estimation scheme that predicts the location of the intruder given a set of UGS measurements has been integrated with human interface software developed by AFRL/RH • Flight test algorithm (Spring ‘13) and demonstrate capability at a joint US/AUS military exercise; Summer 2013 • Three flight tests have validated hardware and software architecture to implement intruder capture algorithms
- 11. 11DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution NETWORKED GUIDANCE AND CONTROL FOR MOBILE MULTI-AGENT SYSTEMS Nuno C. Martins, Univ of Maryland Objective: Tractable methods to design Markov Decision Processes (MDPs) that maximize coverage subject to operational constraints
- 12. 12DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution NETWORKED GUIDANCE AND CONTROL FOR MOBILE MULTI-AGENT SYSTEMS Nuno C. Martins Potential AF impact: - Characterization of the largest configuration set that can be persistently surveyed by a team of autonomous vehicles - Determine the minimum number of vehicles required to survey a certain area, subject to safety and performance constraints. Possible Future directions: - Extension to continuous state-space. - Include output feedback. Scientific impact: - Defined a new optimization paradigm for MDPs, based on maximum entropy methods. locations and directions that can be persistently visited forbidden locations
- 13. 13DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution 2010 MURI (PI: Tamer Başar, UIUC) Multi-Layer and Multi-Resolution Networks of Interacting Agents in Adversarial Environments Vision and Overarching Goal: To address fundamental issues that arise in networked heterogeneous agents, among which are: complex interactions; uncertainty and adversarial actions; trust, learning; humans-in-the-loop; information and communication; and design of architectures to facilitate generation and transmission of actionable information for performance improvement under different equilibrium solutions. MLMR Games: Games played at different levels, interacting through their outcomes, action spaces, and costs “Games within Games”: Zooming in and out providing game structures with different levels of granularity
- 14. 14 Multi-Layer and Multi-Resolution Hybrid Stochastic Differential Games in the Large Population Regime and Their Equilibria • Large-scale interactions happen at different layers of a complex system. Players have inner interactions with members in the group as well as long-range interactions with other groups. • Coupled decisions = Game theory • – Players/Actors/Agents • – Actions/Strategies/Choices • – Preferences over joint choices (utility functions) • – Solution concept (e.g., Nash equilibrium) • Approach: A multi-layer, multi-resolution (MLMR) game framework DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
- 15. 15 Large Population Game Model • The problem is computationally quite challenging. For the case of large populations, the idea of mean-field representation is used. • Key Intuition from Statistical Mechanics: Extremely complex large population particle systems may have simple continuum limits with distinctive bulk properties. • When the number of players is large, the cost of a player does not explicitly depend on the actions of the other players but rather coupled through the distribution of the agent states m. • Characterization of the Nash equilibrium for this case is a significant result. • The mean-field equilibrium is obtained by solving coupled HJB backward equations together with the generalized FPK equation DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
- 16. 16DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution MURI ‘07: Decision Dynamics in Mixed Teams of Humans and Robots PI: John Baillieul (BU) Toward a Control theory of Planning, Perception, and Reaction • Human decision making – • the effects of peer and other social pressure • Modeling spatial and topological knowledge and defining knowledge acquisition metrics for mixed teams of humans and automata • How perceptual dynamics affect human decision makers • Elements of a formal theory perception-based control
- 17. 17DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Challenges: An encompassing theoretical basis for joint cognitive systems did not exist. • Need to understand how human decisions differ from those of ideal decisions makers due to: biases, fear of disapproval, and other forms of group pressure. (Psychology addresses such questions.) • Paradigm shift: humans operating on parity with automatons. Approaches and Results: •Design of reward-incentivized games for play by individual subjects and by groups of subjects, with various forms of information sharing. •Trade-offs between exploitation of strategies with known payoff and exploration of alternative strategies in the hope of higher payoff. • Shared information about reward sizes tended to encourage exploration of strategy alternatives in games that were studied. •When people play with awareness of others playing simultaneously, competitiveness emerges. • Knowledge of others’ reward affects play style more than knowledge of others’ play strategies A deeper understanding of decision making in mixed human/robot teams can be pursued through research on information acquisition, sharing and propagation in complex networks. Decision Making in Mixed Human/Robot Teams
- 18. 18DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Task Partitioning in Mixed Human/Robot Teams •Challenges: •Understand task partitioning between machines and humans in search and reconnaissance. • Use stochastic control principles to develop automated task partitioning and adaptive resource assignment algorithms for teams of agents with finite resources. • Determine advantages/disadvantages of alternative interaction structures between human/automata. •Results: •Automated task partitioning and scheduling algorithms that incorporate risk of vehicle loss; formulated as a stochastic combinatorial problem on graphs. • Dynamic Search/Classify Problems: Agents’ actions generate noisy observations, processed through Bayesian techniques, that are then used by agents in selecting future actions. • Normative solution of such problems requires full stochastic dynamic programming; constraint relaxation allows scalable solution A fundamental distinction between humans and robots in mixed teams is that robots react rapidly to real-time sensor data, whereas humans react to more complex perceptions of the environment in which cognitive processes and prior experience play a role. :
- 19. 19DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Computational Control Theory McEneaney (UCSD) ,Dower (Uni-Melbourne) • Challenges: Develop new theory and computational tools to solve previously intractable optimal control and dynamic game problems. • Approach: 1) Develop and exploit new and existing theory connecting idempotent algebra and dynamic programming to identify or construct fast solution propagators for specific classes of optimal control and dynamic game problem. • 2) Develop new computational tools to implement these fast solution propagators to solve practical problems. • The N-body problem stands as a benchmark in astrodynamics. • The focus lies in the (harder) two-point boundary value N-body problem: – Given a combination of initial and some terminal data, determine the N-body trajectory meeting these conditions. • Applications: Asteroid danger estimation, Efficient space vehicle trajectory design, Concept generalizable to other conservative systems and potentials.
- 20. 20DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Least Action and Game-Based Gravitational Potential Representation • Max-plus based fundamental solutions for Riccati ODE and PDE are immediately transferable to least-action approach to two-point boundary values problems (TPBVPs) with quadratic potentials (e.g., mass-spring, wave equation dynamics). • Least-Action Principle: Conservative system evolves so as to minimize (more correctly find stationary point) of the action (kinetic minus potential energy). • Re-interpretation of the gravitational potential energy. Consider two bodies separated by distance, r. – Traditional potential energy representation: – Competing controller representation: – Note quadratic form for potential as function of position – computationally very useful.
- 21. 21DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Fundamental Solutions => Extremely Rapid Evaluations • Fundamental solution given as a set of solutions of Riccati equations, S(t). (Least action is obtained as max of linear function over convex set S(t).) • Having a fundamental solution allows for very rapid solution of large numbers of TPBVPs (fixed set of masses and duration). • Suppose we are given all N initial positions, x, and N terminal velocities, vf . Want initial velocities (and trajectories). • Repeat ad nauseum for variety of x, vf . S(t) depicted as layers indexed by γ, two-body case. (=initial velocities)
- 22. 22DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution Summary • Dynamics and Control is the key enabler in the science of autonomy. There have been fruitful collaborations with other disciplines such ad cognitive science, network science, communication, information science. – Plan is to initiate new MURI and AFOSR Basic Research Initiative (BRI) topics in V&V, Human-Machine interaction, Autonomy in contested environments • Close collaboration and sharing of information with other agencies: ONR, ARO and NSF • New application areas have benefited from control theoretic approaches, and control theory has grown and changed because of these applications=> Examples: Synthetic biology (new BRI topic), Quantum Control (plan for a future BRI) • Support of fundamental research in Control and Dynamics while considering challenging and relevant application areas is what distinguishes this portfolio.