![00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• What is a deep learning model?
-A directed acyclic graph
Multiple processing units (nodes of the graph) with linear / non-linear transformations
-Model high-level abstractions of the data through processing layers
cs231n, Andrej Karpathy
• How to force the model to discover and efficiently internalize the essence
of the data in order to generate it?
-Use generative models
It works (somehow) for 2D images.
Unsupervised Representation Learning
with Deep Convolutional Generative
Adversarial Networks
[Radford et al, 2015]
Not real images!](https://image.slidesharecdn.com/9420e59d-9b1c-4d47-bdc5-6e35c7cd28d9-160826195238/85/Final_Talk_Tool_Team-20-320.jpg)
![00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Reliable and fast 3D shape
generation
• Our idea:
-Develop deep generative models to output 3D asteroid shapes
-Recover these structures from radar images via probabilistic inference
-Recent work from Google DeepMind
Unsupervised Learning of 3D Structure from Images, [Rezende et al. 2016.]
-Using Voxels representation, Variational Auto-Encoder
-3D shape as output](https://image.slidesharecdn.com/9420e59d-9b1c-4d47-bdc5-6e35c7cd28d9-160826195238/85/Final_Talk_Tool_Team-21-320.jpg)
![00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Next Steps
• Bayesian Optimisation
-New metrics instead of Chi Square
Having a minimum Chi square does not guarantee a physically accurate shape
Work with Naidu Shantanu (NASA JPL) and Michael Busch (SETI)
-Investigate bayesian optimisation with spherical harmonics model
-Develop a full web user interface
Simply upload the radar images and metadata
• Deep Generative Networks
-Try with different generative models
DCGAN (Deep Convolutional Generative Adversarial Networks) [Radford et al, 2015]
PixelRNN (Pixel Recurrent Neural Networks) [Oord et al, 2016]
-Add more parameters in the input (ephemeris, baud length, bandwidth)](https://image.slidesharecdn.com/9420e59d-9b1c-4d47-bdc5-6e35c7cd28d9-160826195238/85/Final_Talk_Tool_Team-30-320.jpg)
Final_Talk_Tool_Team
- 1. EMETTEUR - NOM DE LA PRESENTATION 00 MOIS 2011- New Tools Team Mehdi Lamee Olorato Mosiane Corinne Vassallo Chedy Raissi Mentor: Michael W. Busch
- 2. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Team New Tools Mehdi Lamee Phd candidate University of Minnesota Olorato Mosiane Msc North West University, South Africa Corinne Vassallo Phd Candiate University of Texas, Austin Chedy Raïssi Research Scientist INRIA, France
- 3. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Motivation • Asteroid Grand Challenge -Find all asteroid threats to human populations and know what to do about them Detect Track Characterize Mitigate Communicate • Global effort -Innovative solutions -Accelerate the completion of the survey of main belt and potentially hazardous asteroids
- 4. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Toutatis Itokawa Golevka Motivation: Asteroid shape modeling • Planetary scientists need precise shape models and extra parameters to compute precise orbits and plan safe robotic landings Credits: Ian Webster, http://www.asterank.com
- 5. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Existing Modeling Process • Radar Imaging is a powerful technique to obtain information about solar system bodies - SHAPE modeling software uses radar and optical light curve observations to produce 3D model of asteroids (Inverse problem). SHAPE inversion software 1992UY4, Nick Duong. ?
- 6. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB SHAPE Software: An Overview Generate 3D model from radar images Radar images Project into simulated radar images Compute residuals Vary Param eters
- 7. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Motivation • Existing Modeling Process: 1992 UY4 -processed by Nick Duong (REU Student @SETI) Form at observation files 1 day Fitbasicm odel 1 day M anualgrid search 3 days Fitspherical harm onicsm odel 3 days Fitting vertices m odels 3 weeks 4 weeks!
- 8. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Context • We wish to improve the speed and automation of asteroid shape modeling -Two of the main time intensive steps: 1) The human-controlled manual grid search of spin axis orientations 2) Reliable and fast 3D shape generation (pre-process for SHAPE) How to improve the speed and automation? Spin axis search Fast 3D shape generation Deep Generative models Bayesian Optimization
- 9. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Motivation • Existing Modeling Process: 1992 UY4 -processed by Nick Duong (REU Student @SETI) Form at observation files 1 day Fitbasicm odel 1 day M anualgrid search 3 days Fitspherical harm onicsm odel 3 days Fitting vertices m odels 3 weeks { Bayesian Optimization A few hours { Deep Generative Models Inference: few seconds 1.5 week! Refine com plex vertexm odel 1 week
- 10. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Context How to improve the speed and automation? Spin axis search Fast 3D shape generation Deep Generative models Bayesian Optimization
- 11. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Bayesian Optimization: What are we trying to find? β λ Latitude Longitude
- 12. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Bayesian Optimization • Automate and reduce search area • Global optimization rather than local • Deals with unknown non-convex functions with no derivatives
- 13. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Bayesian Optimization 1. Start with some assumptions (prior) for the distribution of model parameters 2. Use the observations to predict a new distribution for the model parameters (posterior) 3. Decide where to sample next in the parameter space to have the maximum improvement in optimization Acquisition function 4. Posterior becomes the new prior for the next iteration
- 14. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Bayesian Optimization https://github.com/fmfn/BayesianOptimization
- 15. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Bayesian Optimization
- 16. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Results • Teams’ Results: 2008 EV5 -Runtime: ~24 hours - Best parameters Longitude: 255º Latitude: -75º χ2: ~1.641 -Michael Busch parameters: ~3 days Longitude: 270º Latitude: -84º χ2: ~1.678
- 17. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Results • Teams’ Results: 1992 UY4 -Runtime: ~6 hours - Best parameters Longitude: 0º Latitude: 180º χ2: ~1.43 -Nick Duong’s parameters: ~3 days Longitude: 15º Latitude: 170º χ2: ~1.60 1992UY4
- 18. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Results • Teams’ Results: 2000 RS11 -Runtime: ~4 hours -Best parameters: Longitude: 270º Latitude: 45º χ2: ~0.74463 -Kaley Brauer parameters: ~3 days Longitude: 245º Latitude: 60º χ2: ~0.745825
- 19. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Context How to improve the speed and automation? Spin axis search Fast 3D shape generation Deep Generative models Bayesian Optimization
- 20. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • What is a deep learning model? -A directed acyclic graph Multiple processing units (nodes of the graph) with linear / non-linear transformations -Model high-level abstractions of the data through processing layers cs231n, Andrej Karpathy • How to force the model to discover and efficiently internalize the essence of the data in order to generate it? -Use generative models It works (somehow) for 2D images. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks [Radford et al, 2015] Not real images!
- 21. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Reliable and fast 3D shape generation • Our idea: -Develop deep generative models to output 3D asteroid shapes -Recover these structures from radar images via probabilistic inference -Recent work from Google DeepMind Unsupervised Learning of 3D Structure from Images, [Rezende et al. 2016.] -Using Voxels representation, Variational Auto-Encoder -3D shape as output
- 22. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Reliable and fast 3D shape generation • BUT: -Deep models (deep neural networks) Need large set of data to represent the vast amount of asteroid shapes and orbits -Our dataset is limited to asteroids that have been observed (light curves and radar imaging) Clearly not enough (less than 1650 models LC + radar!) • First step in our work -Create synthetic data by matching previously modeled asteroids with orbits from different asteroids
- 23. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Dataset Generation: Process • Acquire asteroid shapes: -1620 in total from JPL echo database (radar) and DAMIT database (light curve) -Transform to voxels • Pull ephemeris from JPL’s Horizons: viewed only from Arecibo Observatory More than 60 different orbits • Create fake observation files based on the orbits: -1620 different shapes available -Use different rotation angles • Generation of synthetic radar images -Noise addition • Total synthetic radar images: ~546000
- 24. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • What is the general idea? • We decided to train a Variational Auto-encoder Training as an autoencoder pθ (x z)pθ (z x) Training use maximum likelihood of p(x) given the training data OpenAI blog Latent variables
- 25. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • Training use maximum likelihood of p(x) given the training data -Problem: intractable posterior distribution p_θ (z | x) -Solution: sampling (MCMC) or approximate p(z|x) with q(z|x) Training as an autoencoder pθ (x z)pθ (z x) Training use maximum likelihood of p(x) given the training data Problem: Cannot be calculated: Solution: • MCMC (too costly) • Approximate p(z|x) with q(z|x) pθ (z x) The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function { Encoder {Decoder
- 26. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • Examples for 2D images Variational auto encoders (idea of low dim manifold) Kingma and Welling. “Auto-Encoding Variational Bayes, International Conference on Learning Representations.” ICLR, 2014. arXiv:1312.6114 Examples of successful unfolding (2D àR28x28, R20x26) Frey Face dataset 2000 pictures of Brendan Frey (20x26)
- 27. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • Asteroids generation -Using 3D Convolutional neural networks Roundness Elongation Condition the generation with Radar image Get plausible 3D shapes for asteroids!
- 28. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function The complete auto-encoder qϕ (x z) pθ (x z) Learning the parameters φ and θ via backpropagation Determining the loss function 2D Convolutional NN + GRU 3D Convolutional VAE At training timeAt inference time
- 29. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB 3D Deep Generative Models • Using Keras framework with Theano backend -2 Nvidia TitanX used for training • Preliminary results: Input: Output: Real shape: Object 2002 CE26 alphaObject 1992 SK
- 30. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Next Steps • Bayesian Optimisation -New metrics instead of Chi Square Having a minimum Chi square does not guarantee a physically accurate shape Work with Naidu Shantanu (NASA JPL) and Michael Busch (SETI) -Investigate bayesian optimisation with spherical harmonics model -Develop a full web user interface Simply upload the radar images and metadata • Deep Generative Networks -Try with different generative models DCGAN (Deep Convolutional Generative Adversarial Networks) [Radford et al, 2015] PixelRNN (Pixel Recurrent Neural Networks) [Oord et al, 2016] -Add more parameters in the input (ephemeris, baud length, bandwidth)
- 31. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION - D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB Next Steps • Project sustainability -Push the code and documentation as an open source project over Scientific packaging -Open Data: Share the generated data (synthetic radar images and shapes) Increase visibility of the project through a challenge? Successful data science challenge: https://www.kaggle.com/c/higgs-boson)
- 32. EMETTEUR - NOM DE LA PRESENTATION 00 MOIS 2011- CONTACT INFO D E F E N S E ACCELERATOR PLANETARY D E V E L O P M E N T FRONTIER LAB THANK YOU