![Andreas Scheidegger – Eawag
Spatial transformer
convolution
Pt
Ht
Pt+1
Ht+1
+
fully
connected
fully
connected
fully
connected
spatial
transformer
deconvolution
^
Gaussian blur
fully
connected
Jaderberg,M.,Simonyan,K.,Zisserman,A.,andKavukcuoglu,K.
(2015)SpatialTransformerNetworks.arXiv:1506.02025[cs].](https://image.slidesharecdn.com/scheideggergeomlarnn-160701144326/85/Recurrent-Neuronal-Network-tailored-for-Weather-Radar-Nowcasting-11-320.jpg)
![Andreas Scheidegger – Eawag
Spatial Transformer
input output
Jaderberg, M., Simonyan, K., Zisserman, A., and
Kavukcuoglu, K. (2015) Spatial Transformer Networks.
arXiv:1506.02025 [cs].](https://image.slidesharecdn.com/scheideggergeomlarnn-160701144326/85/Recurrent-Neuronal-Network-tailored-for-Weather-Radar-Nowcasting-12-320.jpg)
![Spatial Transformer
Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015) Spatial
Transformer Networks. arXiv:1506.02025 [cs].](https://image.slidesharecdn.com/scheideggergeomlarnn-160701144326/85/Recurrent-Neuronal-Network-tailored-for-Weather-Radar-Nowcasting-45-320.jpg)
Recurrent Neuronal Network tailored for Weather Radar Nowcasting
- 1. Eawag: Swiss Federal Institute of Aquatic Science and Technology Recurrent Neuronal Network tailored for Weather Radar Nowcasting June 23, 2016 Andreas Scheidegger
- 2. Andreas Scheidegger – Eawag Weather Radar Nowcasting t0 t-1t-2t-3 t-4 t5t4t3t2 t1 nowcast model available inputs: every pixel of the previous images desired outputs: every pixel of the next n images
- 3. Andreas Scheidegger – Eawag Recurrent ANN Pt Ht Ht+1 ANN Pt+1 ^ last observed image
- 4. Andreas Scheidegger – Eawag Recurrent ANN Pt Ht Ht+1 ANN Ht+2 ANN Pt+1 ^ Pt+2 ^ last observed image
- 5. Andreas Scheidegger – Eawag Recurrent ANN Pt Ht Ht+1 ANN Ht+2 Ht+3 Pt+3 ^ ANN ANN Pt+1 ^ Pt+2 ^ last observed image
- 6. Andreas Scheidegger – Eawag Recurrent ANN Pt Ht Ht+1 ANN Ht+2 Ht+3 Ht+4 Pt+3 ^ Pt+4 ^ ANN ANN ANN Pt+1 ^ Pt+2 ^ last observed image
- 7. Andreas Scheidegger – Eawag Model structure Pt Ht Ht+1 ANN Pt+1 ^ v j=σ(∑ k wjk uk +bj) Traditional Artificial Neuronal Networks (ANN) are (nested) non- linear regressions: Traditional Artificial Neuronal Networks (ANN) are (nested) non- linear regressions: How can we do better?
- 8. Andreas Scheidegger – Eawag Model structure Pt Ht Ht+1 ANN Pt+1 ^ v j=σ(∑ k wjk uk +bj) Traditional Artificial Neuronal Networks (ANN) are (nested) non- linear regressions: Traditional Artificial Neuronal Networks (ANN) are (nested) non- linear regressions: How can we do better? tailor model structure to problem
- 9. Andreas Scheidegger – Eawag Model structure Pt Ht Ht+1 ANN Pt+1 ^
- 10. Andreas Scheidegger – Eawag Model structure convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected
- 11. Andreas Scheidegger – Eawag Spatial transformer convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected Jaderberg,M.,Simonyan,K.,Zisserman,A.,andKavukcuoglu,K. (2015)SpatialTransformerNetworks.arXiv:1506.02025[cs].
- 12. Andreas Scheidegger – Eawag Spatial Transformer input output Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015) Spatial Transformer Networks. arXiv:1506.02025 [cs].
- 13. Andreas Scheidegger – Eawag Spatial transformer convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected
- 14. Andreas Scheidegger – Eawag Local correction convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected
- 15. Andreas Scheidegger – Eawag Local correction
- 16. Andreas Scheidegger – Eawag Local correction
- 17. Andreas Scheidegger – Eawag Gaussian blur convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected
- 18. Andreas Scheidegger – Eawag Recurrent ANN Pt Ht Ht+1 ANN Ht+2 Ht+3 Ht+4 Pt+3 ^ Pt+4 ^ ANN ANN ANN Pt+1 ^ Pt+2 ^ last observed image
- 19. Andreas Scheidegger – Eawag Model structure convolution Pt Ht Pt+1 Ht+1 + fully connected fully connected fully connected spatial transformer deconvolution ^ Gaussian blur fully connected
- 20. Andreas Scheidegger – Eawag Prediction Forecast horizon: 2.5 minutes observed predicted
- 21. Andreas Scheidegger – Eawag Prediction observed predicted Forecast horizon: 15 minutes
- 22. Andreas Scheidegger – Eawag Prediction observed predicted Forecast horizon: 30 minutes
- 23. Andreas Scheidegger – Eawag Prediction observed predicted Forecast horizon: 45 minutes
- 24. Andreas Scheidegger – Eawag Prediction observed predicted Forecast horizon: 60 minutes
- 25. Andreas Scheidegger – Eawag Prediction
- 26. Andreas Scheidegger – Eawag Conclusions
- 27. Andreas Scheidegger – Eawag Deep learning may be a hype – but it's a useful tool nevertheless! Conclusions
- 28. Andreas Scheidegger – Eawag v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling Deep learning may be a hype – but it's a useful tool nevertheless! Conclusions
- 29. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling Deep learning may be a hype – but it's a useful tool nevertheless! Conclusions
- 30. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling θt+1=θt−λ ∇ L(θt) Online parameter adaptionDeep learning may be a hype – but it's a useful tool nevertheless! Conclusions
- 31. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling θt+1=θt−λ ∇ L(θt) Online parameter adaption L(θ)Other objective function? Deep learning may be a hype – but it's a useful tool nevertheless! Conclusions
- 32. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling θt+1=θt−λ ∇ L(θt) Online parameter adaption L(θ)Other objective function? Deep learning may be a hype – but it's a useful tool nevertheless! Include other inputs Conclusions
- 33. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling θt+1=θt−λ ∇ L(θt) Online parameter adaption L(θ)Other objective function? Deep learning may be a hype – but it's a useful tool nevertheless! Include other inputs Predict prediction uncertainty Conclusions
- 34. Andreas Scheidegger – Eawag and Outlook v j=σ( ∑k wjk u k +b j) Combine domain knowledge with data driven modeling θt+1=θt−λ ∇ L(θt) Online parameter adaption L(θ)Other objective function? Deep learning may be a hype – but it's a useful tool nevertheless! Include other inputs Interested in collaboration? andreas.scheidegger@eawag.ch Predict prediction uncertainty Conclusions
- 35. Andreas Scheidegger – Eawag
- 36. Andreas Scheidegger – Eawag Training and Implementaion θnew=θold−λ ∇ L(θold) Loss function Scheduled Learning Regularisation Drop-out L(θ)=∫ 0 ∞ RMS(τ;θ) p(τ)d τ Runs on cuda compatible GPUs Based on Python library chainer Srivastava, et al. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1929–1958. Stochastic Gradient Decent Implementation Bengio, et al. (2015). Scheduled sampling for sequence prediction with recurrent neural networks. arXiv preprint arXiv:1506.03099. Tokui, et al., (2015). Chainer: a Next-Generation Open Source Framework for Deep Learning.
- 37. Andreas Scheidegger – Eawag Training Pt Ht Ht+1 Pt+1 ANN Ht+2 Ht+3 Ht+4 Pt+3 ^ Pt+4 ^ ANN ANN ANN Pt+2 training predictions observed images predicted images
- 38. Andreas Scheidegger – Eawag Scheduled Training Pt Ht Ht+1 Pt+1 ANN Ht+2 Ht+3 Ht+4 Pt+3 ^ Pt+4 ^ ANN ANN ANN Pt+1 ^ or Pt+2 Pt+2 ^ or training predictions observed or predicted images predicted images Bengio, Set al. (2015). Scheduled sampling for sequence prediction with recurrent neural networks. arXiv preprint arXiv:1506.03099.
- 39. Andreas Scheidegger – Eawag Results
- 40. Andreas Scheidegger – Eawag Prediction animated
- 41. Andreas Scheidegger – Eawag Prediction animated
- 42. Andreas Scheidegger – Eawag Traditional model structures Estimate velocity field and translate last radar image (optical flow) Identify rain cells, track and extrapolate them → seems to be the most “natural” approach → can model grows and decay of cells ● Models can be difficult to tune ● Local features (e.g. mountain range) cannot be modeled
- 45. Spatial Transformer Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015) Spatial Transformer Networks. arXiv:1506.02025 [cs].