Deep learning: the final frontier for time series analysis and signal processing?
4 likes410 views
- Deep learning, specifically convolutional neural networks and recurrent neural networks, provides alternatives to traditional time series and signal processing techniques that can learn patterns without specifying a model. - CNNs can learn local geometrical patterns in time series while retaining information over time. RNNs can learn long-range dependencies in high-dimensional data. - Deep learning approaches like autoencoders are well-suited for tasks like pattern matching, anomaly detection, and simulation/generation of time series. Hybrid approaches that combine hand-crafted features with deep learning may also be effective.
Deep learning: the final frontier for time series analysis and signal processing?
- 1. Deep learning: the final frontier for time series analysis and signal processing? Alex Honchar, PyCon Italy 19’
- 2. ML Outsourcing Healthcare Finance ML Architect and cofounder @ Mawi Band Researcher, blogger @ UNIVR, Medium CTO and cofounder @ Neurons Lab
- 10. Signals in the wild: humans
- 11. Signals in the wild: businesses
- 12. Signals in the wild: nature
- 14. Sales time series ECG time series
- 15. Time domain analysis Statistical features Very limited point of view Geometrical features Overcomplex algorithmic heuristics Decompositions Oversimplified econometric POV “ARIMA”-like models Just autocorrelation on different lags Multilayer perceptrons Universal approximation theorem Designed with non-linearity Convolutional neural networks Can learn arbitrary local geometrical patterns
- 16. EEG time series Accelerometer time series
- 17. Frequency domain analysis Fourier transform Very limited point of view Losing information that varies over time Wavelet transform Fixed wavelet family Convolutional neural networks Convolution theorem for frequency analysis Learnable and extendable kernel family Keeps information over time
- 18. Financial time series Econometric time series
- 19. State space models Hidden Markov Models Difficult to train, not for high-dimensional data No long-term dependencies Dynamical factor models Just a vector autoregression? Kalman filters Requires a model of the system Recurrent neural networks Designed to learn long-range dependencies Designed to deal with high-dimensional data Hierarchical state space Non-linearity
- 20. State space models Hidden Markov Models Difficult to train, not for high-dimensional data No long-term dependencies Dynamical factor models Just a vector autoregression? Kalman filters Requires a model of the system Recurrent neural networks Truncated implementation Difficult to optimize Slow in inference Autoregressive networks Connection to autoregressive models Still long-range context Faster and more accurate
- 22. Dynamical systems Differential equations Designed individually for a problem Neural ordinary differential equations Learns the ODE from the data by itself
- 23. Other problems
- 24. Pattern matching Anomaly detection Simulation
- 25. Pattern matching “Normal” distances Don’t work with time series Dynamic time warping Computationally difficult Metric learning Requires a lot of customization Autoencoders Flexibility in choosing encoding scheme Fully unsupervised After training just forward pass to get embedding Siamese networks Learn pattern matching or clustering directly
- 26. Anomaly detection Density-based methods Don’t really work with time series Correlation-based methods Assuming linear properties Fit “ARIMA”, check for residuals Depends on a simple model Autoencoders Flexibility in choosing encoding scheme Fully unsupervised Just need to adjust the thresholds Generative adversarial networks Get a generative model “for free” Use discriminator as anomaly detector
- 28. Simulation and generation Mathematical models … Sequence2sequence schemes Flexibility in choosing encoding scheme Variational autoencoders Object manipulation via disentangled representations Generative adversarial networks State of the art results at the moment Neural ODEs Naturally model dynamical systems with arbitrary precision
- 30. Success stories Sales forecasting Wikipedia traffic Stanford ECG Uber forecasting Reading mindsWavenet
- 31. Success stories
- 32. Fail stories? Statistical and Machine Learning forecasting methods: Concerns and ways forward
- 33. Hybrid solutions Time-series Extreme Event Forecasting with Neural Networks at Uber
- 34. - Time series and signals are everywhere - To all “classical” approaches there are “neural” alternatives - CNN or autoregressive CNN is a baseline - Try unsupervised learning for better embedding space - Simulation with VAEs and GANs is amazing! - Try to combine your hand-crafted features with DL Takeaways
- 35. Open for collaborations :) Facebook / Instagram @rachnogstyle Medium @alexrachnog Linkedin Alexandr Honchar
- 36. Resources * Is Deep Learning the Final Frontier and the End of Signal Processing - Panel Discussion at Technion https://www.youtube.com/watch?v=LZnAFO5gkOQ&t=9s * Stanford ECG: https://stanfordmlgroup.github.io/projects/ecg/ * Groceries sales forecasting: https://www.kaggle.com/c/favorita-grocery-sales-forecasting * Wikipedia traffic forecasting: https://www.kaggle.com/c/web-traffic-time-series-forecasting * Forecasting at Uber: https://eng.uber.com/tag/forecasting/ * DeepMind WaveNet: https://deepmind.com/blog/wavenet-generative-model-raw-audio/ * EEG2Thoughts: https://medium.com/@justlv/using-ai-to-read-your-thoughts-with-keras-and-an- eeg-sensor-167ace32e84a * ECG interpretation: https://medium.com/mawi-band/how-ai-based-arrhythmia-detector-can- explain-its-decisions-b4f433faa4a2 * Replacing mathematical models with NNs in biosignal analysis: https://medium.com/mawi-band/ towards-ai-based-only-biosignal-analysis-pipeline-39e6e31244a6 * Statistical and Machine Learning forecasting methods: Concerns and ways forward: http:// journals.plos.org/plosone/article?id=10.1371/journal.pone.0194889 * Time-series Extreme Event Forecasting with Neural Networks at Uber: http://roseyu.com/time- series-workshop/submissions/TSW2017_paper_3.pdf