Crash course in control theory for neuroscientists and biologists
- 1. Control-theoretic approach to the analysis and synthesis of sensorimotor loops A few main principles and connections to neuroscience Neurotheory and Engineering seminar - 05/28/2013 Matteo Mischiati 1
- 2. • Control theory framework - linear time-invariant (LTI) case • Properties of feedback - internal model principle • Common control schemes - forward and inverse models, Smith predictor - state feedback, observers, optimal control • A model of human response in manual tracking tasks (Kleinman et al. 1970, Gawthrop et al. 2011) 2Matteo Mischiati Control theory primer
- 3. Control theory framework 𝑢 = 𝑖𝑛𝑝𝑢𝑡 (𝑚𝑜𝑡𝑜𝑟) 𝑦 = 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝒙 = 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑠𝑡𝑎𝑡𝑒 𝑑 = 𝑑𝑖𝑠𝑡𝑢𝑟𝑏𝑎𝑛𝑐𝑒 𝑛 = 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑛𝑜𝑖𝑠𝑒 Assume you have a system (PLANT) for which you can control certain variables (inputs) and sense others (outputs). There may be disturbances on your inputs and your sensed outputs may be noisy. 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 𝑢 𝑦 PLANT 𝑢 𝑐 ++ 𝑛 ++ 𝑑 𝑦 𝑛 3Matteo Mischiati Control theory primer
- 4. Control theory framework 𝑢 = 𝑖𝑛𝑝𝑢𝑡 (𝑚𝑜𝑡𝑜𝑟) 𝑦 = 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝒙 = 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑠𝑡𝑎𝑡𝑒 𝑑 = 𝑑𝑖𝑠𝑡𝑢𝑟𝑏𝑎𝑛𝑐𝑒 𝑛 = 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑛𝑜𝑖𝑠𝑒 SYNTHESIS problem: design a controller that applies the right inputs to the plant, based on the noisy outputs, to achieve a desired goal while satisfying one or more performance criteria 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 𝑢 𝑦 PLANT 𝑢 𝑐 ++ 𝑛 ++ 𝑑 𝑦 𝑛𝑦 𝒛 = ℎ 𝒛, 𝑦, 𝑦𝑛 𝑢 𝑐 = 𝑖 𝒛, 𝑦, 𝑦 𝑛 CONTROLLER 4Matteo Mischiati Control theory primer
- 5. Control theory framework 𝑢 = 𝑖𝑛𝑝𝑢𝑡 (𝑚𝑜𝑡𝑜𝑟) 𝑦 = 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝒙 = 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑠𝑡𝑎𝑡𝑒 𝑑 = 𝑑𝑖𝑠𝑡𝑢𝑟𝑏𝑎𝑛𝑐𝑒 𝑛 = 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑛𝑜𝑖𝑠𝑒 SYNTHESIS problem: design a controller that applies the right inputs to the plant, based on the noisy outputs, to achieve a desired goal while satisfying one or more performance criteria Possible Goals: • Output regulation (disturbance rejection, homeostasis) : keep 𝑦 constant despite disturbance • Trajectory tracking : keep 𝑦 𝑡 ≈ 𝑦 𝑡 ∀ 𝑡 Performance criteria: • Static performance (at steady state): e.g. lim 𝑡→∞ 𝑦 𝑡 − 𝑦 𝑡 • Dynamic performance: transient time, etc… • Stability: not blowing up! • Robustness: amount of disturbance that can be tolerated • Limited control effort 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 𝑢 𝑦 PLANT 𝑢 𝑐 ++ 𝑛 ++ 𝑑 𝑦 𝑛𝑦 𝒛 = ℎ 𝒛, 𝑦, 𝑦𝑛 𝑢 𝑐 = 𝑖 𝒛, 𝑦, 𝑦 𝑛 CONTROLLER 5Matteo Mischiati Control theory primer
- 6. Control theory framework 𝑢 = 𝑖𝑛𝑝𝑢𝑡 (𝑚𝑜𝑡𝑜𝑟) 𝑦 = 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝒙 = 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑠𝑡𝑎𝑡𝑒 𝑑 = 𝑑𝑖𝑠𝑡𝑢𝑟𝑏𝑎𝑛𝑐𝑒 𝑛 = 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑛𝑜𝑖𝑠𝑒 ANALYSIS problem: infer the functional structure of the controller, given the observed performance of the overall system in multiple tasks Goals: • Output regulation (disturbance rejection, homeostasis) : keep 𝑦 constant despite disturbance • Trajectory tracking : keep 𝑦 𝑡 ≈ 𝑦 𝑡 ∀ 𝑡 Performance criteria: • Static performance (at steady state): e.g. lim 𝑡→∞ 𝑦 𝑡 − 𝑦 𝑡 • Dynamic performance: transient time, etc… • Stability: not blowing up! • Robustness: amount of disturbance that can be tolerated • Limited control effort 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 ++ 𝑢 𝑛 𝑦 PLANT ++ 𝑑 𝑢 𝑐 𝑦 𝑛𝑦 𝒛 = ℎ 𝒛, 𝑦, 𝑦𝑛 𝑢 𝑐 = 𝑖 𝒛, 𝑦, 𝑦 𝑛 CONTROLLER ? 6Matteo Mischiati Control theory primer
- 7. Example of analysis problem: uncovering the dragonfly control system 𝒗 𝑫𝑭 𝒉𝒆𝒂𝒅 𝒓 • We want to precisely characterize the foraging behavior of the dragonfly (what it does) to gain insight on its neural circuitry (how it does it). 𝒗 𝑫𝑭 𝒉𝒆𝒂𝒅 𝒇𝒍𝒚 ? 𝒓, relative to dragonfly, in 𝒉𝒆𝒂𝒅 ref. frame dragonfly accel. head rotation dragonfly head, body & wing dynamics dragonfly visual system 𝐑𝐄𝐓𝐈𝐍𝐀 ? 𝐓𝐒𝐃𝐍s ? 𝐖𝐈𝐍𝐆 𝐌𝐔𝐒𝐂𝐋𝐄𝐒 𝐍𝐄𝐂𝐊 𝐌𝐔𝐒𝐂𝐋𝐄𝐒 𝒃𝒐𝒅𝒚 𝒃𝒐𝒅𝒚 7Matteo Mischiati Control theory primer
- 8. Linear time-invariant systems 𝑢 = 𝑖𝑛𝑝𝑢𝑡 (𝑚𝑜𝑡𝑜𝑟) 𝑦 = 𝑜𝑢𝑡𝑝𝑢𝑡 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝒙 = 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 𝑠𝑡𝑎𝑡𝑒 𝑑 = 𝑑𝑖𝑠𝑡𝑢𝑟𝑏𝑎𝑛𝑐𝑒 𝑛 = 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑛𝑜𝑖𝑠𝑒 𝑃 𝑠 = 𝑠−𝑧1 𝑠−𝑧2 …(𝑠−𝑧 𝑚) 𝑠−𝑝1 𝑠−𝑝2 …(𝑠−𝑝 𝑛) , 𝑧𝑖∈ ℂ = 𝑧𝑒𝑟𝑜𝑠, 𝑝𝑖 ∈ ℂ = 𝑝𝑜𝑙𝑒𝑠 • Stability (of a system) ⇔ 𝑝1, … , 𝑝 𝑛 have negative real part • Performance (of a system): depends on location of poles and zeros • Related to transfer function in frequency domain: 𝑢 𝑡 = sin 𝜔𝑡 ⇒ 𝑦 𝑡 = 𝑃 𝑗𝜔 ⋅ sin(𝜔𝑡 + Φ 𝑃 𝑗𝜔 ) 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 + 𝐷𝑢 ++ 𝑢 𝑛 𝑦 PLANT ++ 𝑑 𝑢 𝑐 𝑦 𝑛𝑦 𝒛 = 𝐻𝒛 + 𝐺 𝑦 𝑢 𝑐 = 𝐼𝒛 + 𝐿 𝑦 CONTROLLER 𝑃(𝑠) ++ 𝑈(𝑠) 𝑁(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠) 𝑌𝑛(𝑠)𝑌(𝑠) 𝐶(𝑠) CONTROLLER Laplace transform Y 𝑠 = 𝑃 𝑠 ∙ 𝑈(𝑠) 8Matteo Mischiati Control theory primer
- 9. Linear time-invariant systems Laplace transform for signals: • Final value theorem : lim 𝑡→∞ 𝑦 𝑡 = lim 𝑠→0 𝑠 ∙ 𝑌 𝑠 (if limit exists) • If u 𝑡 ⟷ 𝑈 𝑠 , then u 𝑡 ⟷ 𝑈 𝑠 𝑠 Typical reference/disturbance signals: - Step 𝑌 𝑠 = 𝑎 1 𝑠 - Ramp 𝑌 𝑠 = 𝑎 1 𝑠2 - Sinusoid 𝑌 𝑠 = 𝜔 𝑠2+𝜔2 𝑃(𝑠) ++ 𝑈(𝑠) 𝑁(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠) 𝑌𝑛(𝑠)𝑌(𝑠) 𝐶(𝑠) CONTROLLER Y 𝑠 = 𝑃 𝑠 ∙ 𝑈(𝑠) 𝑡 𝑦(𝑡) 𝑎 𝑡 𝑦(𝑡) 𝑎𝑡 𝑡 𝑦(𝑡) sin(𝜔𝑡) 9Matteo Mischiati Control theory primer
- 10. • Control theory framework - linear time-invariant (LTI) case • Properties of feedback - internal model principle • Common control schemes - forward and inverse models, Smith predictor - state feedback, observers, optimal control • A model of human response in manual tracking tasks (Kleinman et al. 1970, Gawthrop et al. 2011) 10Matteo Mischiati Control theory primer
- 11. Feedforward (inverse model) Stability Depends on poles (and zeros!) of 𝑃(𝑠) Performance (static and dynamic) Arbitrarily good if 𝑃 𝑠 ≈ 𝑃 𝑠 and its inverse exists and is stable: Y 𝑠 ≈ 𝑃 𝑠 ∙ 𝑃−1 𝑠 ⋅ 𝑌(𝑠) ≈ 𝑌(𝑠) Robustness to disturbance (disturbance rejection) None : Y 𝑠 = 𝑃 𝑠 ∙ ( 𝑃−1 𝑠 ⋅ 𝑌 𝑠 + 𝐷(𝑠)) ≈ 𝑌(𝑠) + 𝑃(𝑠) ⋅ 𝐷(𝑠) 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠)𝑌(𝑠) 𝑃−1 (𝑠) CONTROLLER Y 𝑠 = 𝑃 𝑠 ∙ 𝑈(𝑠)Uc 𝑠 = 𝑃−1 𝑠 ∙ 𝑌(𝑠) 11Matteo Mischiati Control theory primer
- 12. Properties of Feedback 𝑌 𝑠 = 𝑃 𝑠 𝐷 𝑠 + 𝐶 𝑠 𝐸 𝑠 = 𝑃 𝑠 𝐷 𝑠 + 𝐶 𝑠 ( 𝑌 𝑠 − 𝑌 𝑠 ) ⇒ 𝑌 𝑠 = 𝑃 𝑠 𝐶 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 + 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) Stability Depends on 1 + 𝑃 𝑠 𝐶 𝑆 . Can potentially stabilize unstable plants. Disturbance rejection Depends on 1 + 𝑃 𝑠 𝐶 𝑆 . Can potentially attenuate/cancel effect of 𝐷 𝑠 . 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠) CONTROLLER 𝑌(𝑠) Y 𝑠 = 𝑃 𝑠 ∙ 𝑈(𝑠)Uc 𝑠 = 𝐶 𝑠 ∙ 𝐸(𝑠) + - 12Matteo Mischiati Control theory primer
- 13. Properties of Feedback Static performance lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 𝐸 𝑠 , 𝐸 𝑠 = 1 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 − 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) Let’s see how different controllers perform: 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠) CONTROLLER 𝑌(𝑠) + - 𝑒. 𝑔. 1 1 + 𝜏𝑠 𝑘 13Matteo Mischiati Control theory primer
- 14. Properties of Feedback Static performance lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 𝐸 𝑠 , 𝐸 𝑠 = 1 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 − 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) Proportional controller: 𝐶 𝑠 = 𝑘 lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 1 1+𝑘 𝑃 𝑠 𝑎 𝑠 = 𝑎 1+𝑘 ≠ 0 errors in tracking a step (but small if 𝑘 is large) lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 1 1+𝑘 𝑃 𝑠 𝑎 𝑠2 → ∞ cannot track a ramp at all 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠) CONTROLLER 𝑌(𝑠) + - 𝑎 𝑠 𝑎 𝑠2 𝑒. 𝑔. 1 1 + 𝜏𝑠 𝑘 14Matteo Mischiati Control theory primer
- 15. Properties of Feedback Static performance lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 𝐸 𝑠 , 𝐸 𝑠 = 1 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 − 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) Proportional+Integral (PI) controller: 𝐶 𝑠 = 𝑘 𝑝 + 𝑘 𝑖 𝑠 = 𝑘 𝑝 𝑠+𝑘 𝑖 𝑠 lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 1 1+ 𝑘 𝑝 𝑠+𝑘 𝑖 𝑠 𝑃 𝑠 𝑎 𝑠 = 0 perfect in tracking a step lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 1 1+ 𝑘 𝑝 𝑠+𝑘 𝑖 𝑠 𝑃 𝑠 𝑎 𝑠2 = 𝑎 𝑘 𝑖 ≠ 0 errors in tracking a ramp 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝐷(𝑠) 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠) CONTROLLER 𝑌(𝑠) + - 𝑎 𝑠 𝑎 𝑠2 1 1 + 𝜏𝑠 𝑘 𝑝 + 𝑘𝑖 𝑠 15Matteo Mischiati Control theory primer (but small if 𝑘𝑖 is large)
- 16. Properties of Feedback Static performance lim 𝑡→∞ 𝑒 𝑡 = lim 𝑠→0 𝑠 ∙ 𝐸 𝑠 , 𝐸 𝑠 = 1 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 − 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) To perfectly track: We need: Step Ramp How about sinusoid? We need: 𝑎 𝑠 𝑎 𝑠2 𝜔 𝑠2 + 𝜔2 𝑃(𝑠)𝐶 𝑠 = 1 𝑠 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ 𝑃(𝑠)𝐶 𝑠 = 1 𝑠2 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ 𝑦 𝑡 = sin 𝜔𝑡 ⇓ 𝑒 𝑡 = 𝐺 𝑗𝜔 ⋅ sin(𝜔𝑡 + Φ 𝐺 𝑗𝜔 ) 𝐺 𝑠 𝑃(𝑠)𝐶 𝑠 = 1 𝑠2 + 𝜔2 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ ⇓ 𝐺 𝑗𝜔 = 1 + 𝑃 𝑗𝜔 𝐶 𝑗𝜔 −1 → 0 16Matteo Mischiati Control theory primer
- 17. Internal model principle To perfectly track: We need: Step Ramp Sinusoid Internal model principle: To achieve asymptotical tracking of a reference signal (rejection of a disturbance signal) via feedback, the controller (or the plant) must contain an “internal model” of the signal. It is a necessary condition, not a sufficient condition (need also stability) 𝑎 𝑠 𝑎 𝑠2 𝜔 𝑠2 + 𝜔2 𝑃(𝑠)𝐶 𝑠 = 1 𝑠 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ 𝑃(𝑠)𝐶 𝑠 = 1 𝑠2 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ 𝑃(𝑠)𝐶 𝑠 = 1 𝑠2 + 𝜔2 ⋅ (𝑃 𝑠 𝐶 𝑠 )′ 17Matteo Mischiati Control theory primer
- 18. Feedback vs. Feedforward Feedback • is needed if plant is unstable or for disturbance rejection • does not require full knowledge of the plant • incorporating the knowledge of possible reference and disturbance signals is very useful (internal model principle) Feedforward • if plant is known, and no disturbance, its performance can’t be beat • no sensory delays 𝑌 𝑠 = 𝑃 𝑠 𝐶 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 + 𝑃 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝐷(𝑠) 𝑌 𝑠 = 𝑃 𝑠 ∙ ( 𝑃−1 𝑠 ⋅ 𝑌 𝑠 + 𝐷(𝑠)) ≈ 𝑌(𝑠) + 𝑃(𝑠) ⋅ 𝐷(𝑠) 18Matteo Mischiati Control theory primer
- 19. • Control theory framework - linear time-invariant (LTI) case • Properties of feedback - internal model principle • Common control schemes - forward and inverse models, Smith predictor - state feedback, observers, optimal control • A model of human response in manual tracking tasks (Kleinman et al. 1970, Gawthrop et al. 2011) 19Matteo Mischiati Control theory primer
- 20. Feedback + Feedforward The feedback controller kicks in only if inverse model is incorrect. 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝑈 𝐹𝐵(𝑠) 𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - 𝑃−1 (𝑠) INVERSE MODEL FEEDBACK 𝑈 𝐹𝐹(𝑠) 20Matteo Mischiati Control theory primer
- 21. Feedback + Feedforward The feedback controller kicks in only if inverse model is incorrect. The corrective command from the feedback path can be also used as a learning/adaptation signal by the inverse model. 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝑈 𝐹𝐵(𝑠) 𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - 𝑃−1 (𝑠) INVERSE MODEL FEEDBACK 𝑈 𝐹𝐹(𝑠) 21Matteo Mischiati Control theory primer
- 22. Feedback + Feedforward The feedback controller kicks in only if inverse model is incorrect. The corrective command from the feedback path can be also used as a learning/adaptation signal by the inverse model. Significant sensory delays are still a problem. 𝑃(𝑠)𝑈(𝑠) 𝑌(𝑠) PLANT ++ 𝑈 𝐹𝐵(𝑠) 𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - 𝑃−1 (𝑠) INVERSE MODEL FEEDBACK 𝑈 𝐹𝐹(𝑠) 𝑒−𝑠𝜏 22Matteo Mischiati Control theory primer
- 23. Forward model The control signal is sent through a model of the plant (“forward model”) to predict the sensory output. 𝑃(𝑠) 𝑌(𝑠) PLANT 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - 𝑃(𝑠) FORWARD MODEL CONTROLLER predicted sensory output 23Matteo Mischiati Control theory primer
- 24. Forward model The control signal is sent through a model of the plant (“forward model”) to predict the sensory output. The (delayed) sensory output can be used as a learning/adaptation signal for the forward model. Direct use of the delayed sensory output in the control is problematic because of time mismatch. 𝑃(𝑠) 𝑌(𝑠) PLANT 𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - 𝑃(𝑠) FORWARD MODEL 𝑒−𝑠𝜏 CONTROLLER predicted sensory output 24Matteo Mischiati Control theory primer
- 25. Smith predictor Assuming 𝑃 𝑠 ≈ 𝑃 𝑠 and 𝜏 ≈ 𝜏: 𝑌 𝑠 = 𝑃 𝑠 𝑈𝑐 𝑠 = 𝑃 𝑠 𝐶 𝑠 1 + 𝑃 𝑠 𝐶 𝑆 𝑌 𝑠 Delay has been moved outside the control loop. PLANT 𝑃(𝑠) 𝑌(𝑠)𝑈 𝐶(𝑠)𝐸(𝑠) 𝐶(𝑠)𝑌(𝑠) + - - 𝑃(𝑠) 𝑒−𝑠𝜏 𝑒−𝑠 𝜏 + - delay model plant model predicted sensory output error in sensory output prediction CONTROLLER 25Matteo Mischiati Control theory primer
- 26. Models of the cerebellum 1. Cerebellum as an inverse model in a feedback+feedforward motor control scheme Wolpert, Miall & Kawato, 1998 “Internal models in the cerebellum” Not in the sense of my presentation ! 26Matteo Mischiati Control theory primer
- 27. Models of the cerebellum 2. Cerebellum as a forward model in a Smith predictor control scheme Wolpert, Miall & Kawato, 1998 “Internal models in the cerebellum” 27Matteo Mischiati Control theory primer
- 28. State feedback 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 𝑦 PLANT 𝑢𝑦 𝒛 = ℎ 𝒛, 𝑦, 𝒙 𝑢 = 𝑖 𝒛, 𝑦, 𝒙 CONTROLLER 𝒙 𝑦 PLANT 𝑢𝑦 CONTROLLER 𝒙 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 + 𝐷𝑢 𝒛 = 𝐻𝒛 + 𝐺 𝑦 + 𝑀𝒙 𝑢 = 𝐼𝒛 + 𝐿 𝑦 + 𝑁𝒙 Linear time-invariant case: 28Matteo Mischiati Control theory primer
- 29. State feedback If the plant is reachable, it is possible to achieve any arbitrary choice of closed-loop poles with an appropriate linear and memoryless controller: u = −𝐾𝒙 + 𝐾𝑟 𝑦 𝒙 = 𝑓 𝒙, 𝑢 𝑦 = 𝑔 𝒙, 𝑢 𝑦 PLANT 𝑢𝑦 𝒛 = ℎ 𝒛, 𝑦, 𝒙 𝑢 = 𝑖 𝒛, 𝑦, 𝒙 CONTROLLER 𝒙 𝑦 PLANT 𝑢𝑦 CONTROLLER 𝒙 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 + 𝐷𝑢 Linear time-invariant case: 𝐾 𝐾𝑟 + - 29Matteo Mischiati Control theory primer
- 30. Observers Observer: dynamical system designed to estimate the full state (when not fully available) If the plant is observable, it is possible to achieve lim 𝑡→∞ 𝒙 𝑡 = 𝒙 (with right 𝐿) 𝑦 PLANT 𝑢 𝒙 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 𝒙 = 𝐴 𝒙 + 𝐵𝑢 + 𝐿(y − C 𝒙) OBSERVER 30Matteo Mischiati Control theory primer
- 31. Observers Observer: dynamical system designed to estimate the full state (when not fully available) If the plant is observable, it is possible to achieve lim 𝑡→∞ 𝒙 𝑡 = 𝒙 (with right 𝐿) Separation principle: if the plant is reachable & observable, can replace 𝒙 with 𝒙 and design 𝐾 independently of 𝐿 (use observed state just as real one) 𝑦 PLANT 𝑢 𝒙 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 𝒙 = 𝐴 𝒙 + 𝐵𝑢 + 𝐿(y − C 𝒙) 𝑦 𝐾 𝐾𝑟 +- OBSERVER 31Matteo Mischiati Control theory primer
- 32. Optimal control Linear Quadratic Gaussian (LQG) optimal output regulation: linear plant, additive Gaussian white noise on state (with covariance Σ 𝑑) and output (𝜎 𝑛); minimize quadratic cost : 𝔼 lim 𝑇→∞ 1 𝑇 0 𝑇 𝒙 𝑇 𝑡 𝑄𝒙 𝑡 + 𝑢 𝑡 2 𝑑𝑡 Solution is linear observer (Kalman filter) with linear memoryless controller: 𝐿 = 𝑃𝐶 𝑇 𝜎 𝑛 −1, 𝐴𝑃 + 𝑃𝐴 𝑇 + Σ 𝑑 − 𝑃𝐶 𝑇 𝜎 𝑛 −1 𝐶𝑃 𝑇 = 0 𝐾 = 𝐵 𝑇 𝑆, 𝐴 𝑇 𝑆 + 𝑆𝐴 + 𝑄 − 𝑆𝐵𝐵 𝑇 𝑆 = 0 𝑦 PLANT 𝑢 𝒙 = 𝐴𝒙 + 𝐵𝑢 + 𝒅 𝑦 = 𝐶𝒙 𝒙 𝒙 = 𝐴 𝒙 + 𝐵𝑢 + 𝐿(yn − C 𝒙) 𝑦 = 0 𝐾 +- OBSERVER (KALMAN FILTER) 𝒅 ++ 𝑛 𝑦 𝑛 32Matteo Mischiati Control theory primer
- 33. Internal model principle Internal model principle (state space): to achieve asymptotical tracking of a reference signal (rejection of a disturbance signal) produced by an exosystem, the controller must contain an “internal model” of the exosystem. (Francis & Wonham, Automatica, 1970) It is a necessary condition, not a sufficient condition (need also stability). General principle with extensions to nonlinear systems. 𝑦 PLANT 𝒖𝑦 𝒙 = 𝐴𝒙 + 𝐵𝑢 𝑦 = 𝐶𝒙 𝜼 = 𝑆𝜼 + 𝐺𝑒𝒛 = 𝑆 𝒛 𝑦 = 𝑇 𝒛 + - 𝑒 STABILIZING CONTROLLER 𝜼INT.MODELEXOSYSTEM 33Matteo Mischiati Control theory primer
- 34. • Control theory framework - linear time-invariant (LTI) case • Properties of feedback - internal model principle • Common control schemes - forward and inverse models, Smith predictor - state feedback, observers, optimal control • A model of human response in manual tracking tasks (Kleinman et al. 1970, Gawthrop et al. 2011) 34Matteo Mischiati Control theory primer
- 35. Human response model D. L. Kleinman, S. Baron and W. H. Levison, “An Optimal Control Model of Human Response. Part I: Theory and Validation”, Automatica, 1970 (revisited, more recently, by Gawthrop et al. 2011) Task: by controlling a joystick (position, velocity or acceleration control), subject is asked to keep a cursor on the screen as close as possible to a target location, while unknown disturbances are applied by the computer. Plant: 𝑚𝑜𝑛𝑖𝑡𝑜𝑟 𝑌(𝑠) ++ 𝐷(𝑠) (computer) 𝑈(𝑠) 𝑗𝑜𝑦𝑠𝑡𝑖𝑐𝑘 𝑃𝐽𝑀 𝑠 ∈ 𝑘, 𝑘 𝑠 , 𝑘 𝑠2 𝑠𝑒𝑛𝑠𝑜𝑟𝑦 𝑓𝑒𝑒𝑑𝑏𝑎𝑐𝑘 “Human controller”: 𝑛𝑒𝑢𝑟𝑜𝑚𝑜𝑡𝑜𝑟 dynamics 𝑈(𝑠) ++ 𝑚𝑜𝑡𝑜𝑟 𝑛𝑜𝑖𝑠𝑒 𝑛𝑒𝑢𝑟𝑎𝑙 computation 𝑃 𝑁 𝑠𝐶 𝑠 𝑒−𝑠𝜏 𝑃 𝑁 𝑠 ≈ 1 𝜏 𝑛 𝑠+1 𝜏 𝑛 ≈ 100𝑚𝑠 𝜏 ≈ 150 − 250𝑚𝑠 35Matteo Mischiati Control theory primer
- 36. Human response model Task: Output regulation/disturbance rejection with linear time-invariant plant and significant delay on any potential feedback line 𝑒−𝑠𝜏 𝑃(𝑠) ++ 𝑈(𝑠) 𝑁(𝑠) 𝑌(𝑠) ++ 𝐷(𝑠) 𝑈 𝐶(𝑠) 𝑌𝑛(𝑠)𝐶(𝑠) CONTROLLER 𝑃𝐽𝑀 𝑠 ∙ 𝑃 𝑁(𝑠) ANALYSIS problem: infer a model of the neural controller 𝐶(𝑠) from the observed performance of the subjects tested. 36Matteo Mischiati Control theory primer
- 37. Human response model Task: Output regulation/disturbance rejection with linear time-invariant plant and significant delay on any potential feedback line 𝑒−𝑠𝜏 𝑃(𝑠) ++ 𝑈(𝑠) 𝑁(𝑠) 𝑌(𝑠) ++ 𝐷(𝑠) 𝑈 𝐶(𝑠) 𝑌𝑛(𝑠)𝐶(𝑠) CONTROLLER 𝑃𝐽𝑀 𝑠 ∙ 𝑃 𝑁(𝑠) ANALYSIS problem: infer a model of the neural controller 𝐶(𝑠) from the observed performance of the subjects tested. So what are the performances? • Very good and robust to disturbances up to 2Hz (sum of sinusoids), for all three types of joystick dynamics • Apparently delay-free Must be some kind of FEEDBACK + FORWARD model ! 37Matteo Mischiati Control theory primer
- 38. Human response model Hypothesis: optimal control to minimize average error & control effort 𝔼 lim 𝑇→∞ 1 𝑇 0 𝑇 𝒙 𝑡 2 + 𝛼 𝑢 𝑡 2 + 𝛽 𝑢 𝑡 2 𝑑𝑡 Theoretical solution * (with assumptions similar to LQG problem): - Optimal observer (Kalman filter) to estimate delayed state (as in LQG) - Optimal least mean-squared predictor to predict current state - Optimal linear memoryless controller (as in LQG) 𝑦 PLANT 𝑢 𝒙(𝑡 − 𝜏) 𝒙 = 𝐴𝒙 + 𝐵𝑢 + 𝒅 𝑦 = 𝐶𝒙 KALMAN FILTER 𝑦 = 0 𝐾 +- 𝒅 ++ 𝑛 𝑦 𝑛 * D. Kleinman, “Optimal control of linear systems with time-delay and observation noise”, IEEE Trans. Autom. Control, 1969 𝑒−𝑠𝜏PREDICTOR 𝒙(𝑡) 𝑦𝑛(𝑡 − 𝜏) 38Matteo Mischiati Control theory primer
- 39. Controller freq. response with plant 𝑘 𝑠 Controller freq. response with plant 𝑘 𝑠2 Matteo Mischiati Control theory primer 39
- 40. Human response model Gawthrop et al. * (2011): - Introduced, in both estimator and predictor, a copy of the exosystem generating sinusoidal disturbances (internal model principle!) - Show that intermittent control is also compatible with results * P. Gawthrop et al., “Intermittent control: a computational theory of human control”, Biol. Cybern., 2011 Actual response to sinusoid Response without int.model 40Matteo Mischiati Control theory primer
- 41. • Crash course in control theory (for LTI systems) - many concepts can be extended to more general settings • An example of control-theoretic approach to modeling sensorimotor loops - need to iterate between modeling/experiments to discern among alternatives and improve understanding of the system Conclusions THANK YOU FOR YOUR ATTENTION ! 41Matteo Mischiati Control theory primer
Editor's Notes
- #24: Fast feedback loop involving the forward model can be seen as playing the role of an inverse model.
- #25: Fast feedback loop involving the forward model can be seen as playing the role of an inverse model.
- #33: Here we consider an output regulation problem.
- #34: Example: S=[0 –omega; omega 0] produces oscillatory signals.