20171025 pp-in-robotics
3 likes1,091 views
The document discusses probabilistic programming in robotics. It introduces probabilistic programming and recent advances in probabilistic inference techniques. It then provides an example of using PyMC3 to perform SLAM by formulating the SLAM problem probabilistically and using variational inference to estimate unknown variables like landmark and robot locations. The summary concludes by mentioning some technical issues for applying probabilistic programming to real robotics.
![Confidential
センスタイムジャパンRecent advances in probabilistic inference
• Traditional techniques
§ MCMC – slow for models with many RVs
§ VI for conjugate models – limitation on models,
derivation and implementation of inference
• Advanced techniques
§ VI with stochastic gradient [1] – arbitrary models
§ Automated inference (ADVI) [2] – without
derivation/implementation of inference
§ Auto-encoding VB (VAE) [3] – latent variables
§ Normalizing flows [4], GAN [5] – arbitrary posterior
6](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-6-320.jpg)
![Confidential
センスタイムジャパンWhy PP matters in robotics
• Why complex probabilistic models matter
§ Low-dimensional state representation
§ Incorporation of prior knowledge
§ Composition of multiple models
7
Encoder (VAE) Decoder (VAE)RL
Deep predictive policy architecture for robot manipulation task [6]](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-7-320.jpg)
![Confidential
センスタイムジャパンMotion model
12
Adopted from [7]
• Gaussian fluctuation with a discrete time model:
p(st|st 1, ut) = N(f(st 1, ut), ⌃mot)](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-12-320.jpg)
![Confidential
センスタイムジャパンObservation model
14
• Range-bearing measurement
p(zt|st, M) =
Y
i2D(st)
N(h(mi|st), ⌃obs)
D(s) = {i|distance(mi, s) < threshold}
Adopted from [7]Note: D(s_t) is known here.](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-14-320.jpg)
![Confidential
センスタイムジャパンInference
16
• Mean-field approximation
• Maximization of variational objective (evidence
lower bound; ELBO [1][2][3][4])
q(·): Normal distribution
L(✓) = Eq [ln p(S, M, Z|U)] Eq [ln q(S, M)]
p(S, M|Z, U) ⇡ q(S, M) =
TY
t=1
q(st)
IY
i=1
q(mi)
: (variational) parameters of q(·)✓](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-16-320.jpg)
![Confidential
センスタイムジャパンReferences
20
[1] Paisley, J., Blei, D. M., & Jordan, M. I. (2012, June). Variational
Bayesian inference with stochastic search. ICML 2012.
[2] Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., & Blei, D. M.
(2017). Automatic Differentiation Variational Inference. JMLR 2017.
[3] Kingma, D. P., & Welling, M. (2013). Auto-encoding variational
bayes. ICLR 2014.
[4] Rezende, D., & Mohamed, S. (2015). Variational Inference with
Normalizing Flows. ICML 2015.
[5] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D.,
Ozair, S., & Bengio, Y. (2014). Generative adversarial nets. NIPS 2014.
[6] Ghadirzadeh, A., Maki, A., Kragic, D., & Björkman, M. (2017). Deep
Predictive Policy Training using Reinforcement Learning. IROS 2017.
[7] Tim Bailey (2009). Simultaneous Localisation and Mapping:
Probabilistic Formulation. Presentation slide at SLAM SUMMER SCHOOL
2009, organized by Australian Centre for Field Robotics](https://image.slidesharecdn.com/20171025-pp-in-robotics-171026002222/85/20171025-pp-in-robotics-20-320.jpg)
20171025 pp-in-robotics
- 2. Confidential センスタイムジャパンAbout me • Name: Taku Yoshioka • Interests: Bayesian inference, machine learning, deep learning and robotics • Robot and ROS: 6 months • Affiliation: SenseTime Japan § Computer vision and deep learning § https://www.sensetime.jp § https://blog.sensetime.jp (lunch blog) § Kyoto, Tokyo § We are hiring! 2
- 3. Confidential センスタイムジャパンAgenda • Probabilistic programming (PP) § Bayesian neural network § What is PP § Recent advances in probabilistic inference § Why PP matters in robotics • Example: SLAM with PyMC3 § https://taku- y.github.io/notebook/20170919/slam_advi.html • Technical issues for real robotics application 3
- 4. Confidential センスタイムジャパンBayesian neural network 4 • Two-class classification model with PyMC3 • Left: posterior mean. Right: posterior standard deviation (uncertainty) § http://docs.pymc.io/notebooks/bayesian_neural_ne twork_advi.html
- 5. Confidential センスタイムジャパンWhat is PP • Programming of probabilistic models and inference with high-level API: § Probability distribution, random variables (RVs) § MCMC (Gibbs, HMC), variational inference (VI) § GLM, mixture models, Gaussian processes § Stan, PyMC3, Edward • Traditional application: bioinformatics, finance – exploration of hypothesis (models) • Advances in inference techniques – application with large models (i.e., a large number of RVs) 5
- 6. Confidential センスタイムジャパンRecent advances in probabilistic inference • Traditional techniques § MCMC – slow for models with many RVs § VI for conjugate models – limitation on models, derivation and implementation of inference • Advanced techniques § VI with stochastic gradient [1] – arbitrary models § Automated inference (ADVI) [2] – without derivation/implementation of inference § Auto-encoding VB (VAE) [3] – latent variables § Normalizing flows [4], GAN [5] – arbitrary posterior 6
- 7. Confidential センスタイムジャパンWhy PP matters in robotics • Why complex probabilistic models matter § Low-dimensional state representation § Incorporation of prior knowledge § Composition of multiple models 7 Encoder (VAE) Decoder (VAE)RL Deep predictive policy architecture for robot manipulation task [6]
- 8. Confidential センスタイムジャパンExample: SLAM with PyMC3 8 • Formulation • Simulated data • Motion model • Observation model • Inference • Sampling from approximated posterior • Result
- 9. Confidential センスタイムジャパンFormulation 9 : control signals (known)U = {ut}T t=1 Z = {zt}T t=1 : observations (known) : car locations/directions (unknown) M = {mi}I i=1 : landmark locations (unknown) • 2-D car, landmarks S = {st}T t=0 p(S, M|Z, U) / p(S, M, Z|U) = TY t=1 p(zt|st, M)p(st|st 1, ut)p(s0)p(M) Note: s_0 is fixed in the example.
- 11. Confidential センスタイムジャパンSimulated data 11 • Green trace: prior of the car locations (known) • Red dashed lines: observations of landmarks (known) • Blue trace: true locations of the car (unknown) • Stars: landmarks locations (unknown) • Inference of unknown RVs from knowns
- 12. Confidential センスタイムジャパンMotion model 12 Adopted from [7] • Gaussian fluctuation with a discrete time model: p(st|st 1, ut) = N(f(st 1, ut), ⌃mot)
- 14. Confidential センスタイムジャパンObservation model 14 • Range-bearing measurement p(zt|st, M) = Y i2D(st) N(h(mi|st), ⌃obs) D(s) = {i|distance(mi, s) < threshold} Adopted from [7]Note: D(s_t) is known here.
- 16. Confidential センスタイムジャパンInference 16 • Mean-field approximation • Maximization of variational objective (evidence lower bound; ELBO [1][2][3][4]) q(·): Normal distribution L(✓) = Eq [ln p(S, M, Z|U)] Eq [ln q(S, M)] p(S, M|Z, U) ⇡ q(S, M) = TY t=1 q(st) IY i=1 q(mi) : (variational) parameters of q(·)✓
- 18. Confidential センスタイムジャパンResult 18 • Red trace: posterior mean of states § Improvements from prior (green trace) • Diamonds: estimated locations of 4 landmarks
- 19. Confidential センスタイムジャパンTechnical issues for real robot application 19 • Computational efficiency in prediction § No control over computation on expression graph with backend (Tensorflow, Theano) and Python interpreter § Desired solution: zero-cost abstraction • Optimization in real time • Composition of multiple models • Standard format (e.g., JSON) of probabilistic models for reuse
- 20. Confidential センスタイムジャパンReferences 20 [1] Paisley, J., Blei, D. M., & Jordan, M. I. (2012, June). Variational Bayesian inference with stochastic search. ICML 2012. [2] Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., & Blei, D. M. (2017). Automatic Differentiation Variational Inference. JMLR 2017. [3] Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. ICLR 2014. [4] Rezende, D., & Mohamed, S. (2015). Variational Inference with Normalizing Flows. ICML 2015. [5] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., & Bengio, Y. (2014). Generative adversarial nets. NIPS 2014. [6] Ghadirzadeh, A., Maki, A., Kragic, D., & Björkman, M. (2017). Deep Predictive Policy Training using Reinforcement Learning. IROS 2017. [7] Tim Bailey (2009). Simultaneous Localisation and Mapping: Probabilistic Formulation. Presentation slide at SLAM SUMMER SCHOOL 2009, organized by Australian Centre for Field Robotics