時系列データと確率的プログラミング tfp.sts
2 likes2,879 views
Yuta Kashino is the CEO of BakFoo Inc. and has over 20 years of experience working with Python and related technologies. He discusses structural time series models in TensorFlow Probability and how they can be used for time series forecasting. Key components of structural time series models in TFP include LocalLinearTrend, Sum, build_factored_surrogate_posterier, and fit_surrogate_posterier for variational inference, and fit_with_hmc for MCMC. Structural time series models provide a probabilistic approach to time series modeling and forecasting.
![TFP VI: tfp.vi
❖ sts model
❖ sts.LocalLinearTrend
❖ sts.Sum([trend, seasonal]…)
❖ sts.build_factored_surrogate_posterier
❖ arg: model
❖ vi.fit_surrogate_porterier
❖ arg: model
❖ arg: surrogate_posterier
❖ arg: num_step
❖ sts.forecast](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-9-320.jpg)
![sts model
❖ sts.LocalLinearTrend
❖ sts.Sum([trend, seasonal]…)](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-10-320.jpg)
![sts.Sum([trend, seasonal]…)
❖ sts model
|
t-1
|
t
|
t+1](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-13-320.jpg)
![P(θ|D,m) KL q(θ)
ELBO
⇤
= argmin KL(q(✓; ) || p(✓ | D))
= argmin Eq(✓; )[logq(✓; ) p(✓ | D)]
ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )]
⇤
= argmax ELBO( )
P(✓ | D, m) =
P(D | ✓, m)P(✓ | m)
P(D | m)](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-18-320.jpg)
![- KL =ELBO
- P q
q(✓; 1)
q(✓; 5)
p(✓, D) p(✓, D)
✓✓
⇤
= argmax ELBO( )
ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )]](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-19-320.jpg)
![ELBO
❖ vi.fit_surrogate_porterier
⇤
= argmax ELBO( )
ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )]](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-23-320.jpg)
![TFP MCMC: sts.fit_with_hmc
❖ sts model
❖ sts.Sum([trend, seasonal]…)
❖ sts.fit_with_hmc
❖ model
❖ data](https://image.slidesharecdn.com/bayes2020-200906044727/85/tfp-sts-26-320.jpg)
時系列データと確率的プログラミング tfp.sts
- 1. tfp.sts ( )
- 2. Yuta Kashino ( ) ❖ BakFoo, Inc. CEO ❖ ❖ Kubernetes / ❖ POC / → ❖ ❖ k8s, Web , ❖ Zope 3 / Python 2.1 ❖ Zope Japan ❖ Yahoo! Japan ❖ (~20 ) ❖ ❖ @yutakashino
- 3. ❖ tfp ❖ ❖ ❖ VI/MCMC ❖ CO2
- 4. tfp.sts ❖ Structural Time Series ❖
- 6. ❖ ❖ X0 P0 Xk-1 Pk-1 Xk p = A Xk-1 + B uk + wk Pk p = A Pk-1 AT + Qk Yk = C Xk + zk K = Pk p H / (H Pk p HT + R) Xk = Xk p + K (Yk - Hkp ) Xk Pk = ( I - KH ) Pk p Xk Pk | k-1 | k k -> k-1 ( ) uk : wk : Qk : Yk : zk : K : R : X : P : G(X, P)
- 7. ❖ Michel van Biezen (Raytheon) ❖ https://www.youtube.com/playlist?list=PLX2gX- ftPVXU3oUFNATxGXY90AULiqnWT ❖ 27
- 8. (STS) ❖ Structual Time Series Model ❖ ❖ ❖ : ( ) | t-2 | t-1 | t | t+1 | t+2
- 9. TFP VI: tfp.vi ❖ sts model ❖ sts.LocalLinearTrend ❖ sts.Sum([trend, seasonal]…) ❖ sts.build_factored_surrogate_posterier ❖ arg: model ❖ vi.fit_surrogate_porterier ❖ arg: model ❖ arg: surrogate_posterier ❖ arg: num_step ❖ sts.forecast
- 10. sts model ❖ sts.LocalLinearTrend ❖ sts.Sum([trend, seasonal]…)
- 13. sts.Sum([trend, seasonal]…) ❖ sts model | t-1 | t | t+1
- 14. sts.build_factored_surrogate_posterier ❖ sts.build_factored_surrogate_posterier ❖ return: variational_posterior (tfd.JointDistributionNamed) ❖ ↑ ❖ q
- 15. •- data hypothesis( ) - : - - P(H | D) = P(H)P(D | H) P H P(H)P(D|H) P(x) = X y P(x, y) P(x, y) = P(x)P(y | x) posterior likelihoodprior evidence
- 16. - : - : - - - : P(H | D) = P(H)P(D | H) P H P(H)P(D|H) likelihood priorposterior P(✓ | D, m) = P(D | ✓, m)P(✓ | m) P(D | m) m: P(x | D, m) = Z P(x | ✓, D, m)P(✓ | D, m)d✓ P(m | D) = P(D | m)P(m) P(D) evidence ✓ ⇠ Beta(✓ | 2, 2)
- 17. - (MCMC) - (Variational Inference) P(✓ | D, m) = P(D | ✓, m)P(✓ | m) P(D | m) Z P(D | ✓, m)P(✓)d✓ evidence
- 18. P(θ|D,m) KL q(θ) ELBO ⇤ = argmin KL(q(✓; ) || p(✓ | D)) = argmin Eq(✓; )[logq(✓; ) p(✓ | D)] ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )] ⇤ = argmax ELBO( ) P(✓ | D, m) = P(D | ✓, m)P(✓ | m) P(D | m)
- 19. - KL =ELBO - P q q(✓; 1) q(✓; 5) p(✓, D) p(✓, D) ✓✓ ⇤ = argmax ELBO( ) ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )]
- 20. - P q - : - ADVI: Automatic Differentiation Variational Inference - BBVI: Blackbox Variational Inference arxiv:1603.00788 arxiv:1401.0118 https://github.com/HIPS/autograd/blob/master/examples/bayesian_neural_net.py
- 21. - VI - - David MacKay “Lecture 14 of the Cambridge Course” - PRML 10 http://www.inference.org.uk/itprnn_lectures/
- 22. sts.build_factored_surrogate_posterier ❖ sts.build_factored_surrogate_posterier ❖ return: variational_posterior (tfd.JointDistributionNamed) ❖ q q(✓; 1) q(✓; 5) p(✓, D) p(✓, D) ✓✓
- 23. ELBO ❖ vi.fit_surrogate_porterier ⇤ = argmax ELBO( ) ELBO( ) = Eq(✓; )[p(✓, D) logq(✓; )]
- 24. vi.fit_surrogate_posterier ❖ vi.fit_surrogate_porterier ❖ model ❖ surrogate_posterier ❖ num_step
- 26. TFP MCMC: sts.fit_with_hmc ❖ sts model ❖ sts.Sum([trend, seasonal]…) ❖ sts.fit_with_hmc ❖ model ❖ data
- 27. TFP MCMC: sts.fit_with_hmc ❖ CO2 ❖ 70 ❖ 10 90 ❖ …
- 28. - - P(✓ | D, m) = P(D | ✓, m)P(✓ | m) liklihood priorposterior ✓ https://github.com/dfm/corner.py
- 31. tfp: normal ❖ tfp.python.distributions.normal ❖ https://github.com/tensorflow/probability/blob/v0.11.0/tensorflow_probability/ python/distributions/normal.py ❖ tf.random.normal ❖ https://www.tensorflow.org/api_docs/python/tf/random/normal ❖ tf.python.ops.random_ops ❖ https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/python/ops/ random_ops.py#L43-L98 ❖ tf.python.ops.distributions.distribution.py ❖ https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/python/ops/ distributions/distribution.py ❖ tensorflow::ops::RandomNormal ❖ https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/random-normal ❖ https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/core/ops/ random_ops.cc ❖ https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/core/kernels/ parameterized_truncated_normal_op.cc ❖ https://github.com/tensorflow/tensorflow/blob/v2.3.0/tensorflow/core/kernels/ parameterized_truncated_normal_op.h ❖ Eigen::MatrixExponential
- 32. PyMC3: normal ❖ pymc3.distributions.continuous.normal ❖ https://github.com/pymc-devs/pymc3/blob/master/pymc3/ distributions/continuous.py ❖ scipy.stas.norm ❖ https://docs.scipy.org/doc/scipy/reference/generated/ scipy.stats.norm.html#scipy.stats.norm ❖ scipy.stats._continuous_distns.norm_gen ❖ https://github.com/scipy/scipy/blob/master/scipy/stats/ _continuous_distns.py ❖ np.exp ❖ https://numpy.org/doc/stable/reference/generated/numpy.exp.html ❖ np.core.src.npymath.npy_math_complex.c.src ❖ https://github.com/numpy/numpy/blob/master/numpy/core/src/ npymath/npy_math_complex.c.src#L196
- 33. tfp.sts: pros ❖ tfp.distributions (tfd) ❖ colaboratory ❖ @tf.function(experimental_compile=true) ❖ Bijector API ❖ API ❖ BigQuery
- 34. tfp.sts: cons ❖ MCMC … ❖ ❖ Traceback ❖ API Tensor shape ❖ API ❖ ❖ PyMC4, edword2 … ❖
- 35. FBProphet ❖ Python cmdstan ❖ stan ❖ https://github.com/facebook/prophet/blob/master/python/stan/unix/ prophet.stan ❖ Prophet ❖ https://github.com/facebook/prophet/blob/master/python/fbprophet/ forecaster.py ❖ ❖ https://peerj.com/preprints/3190/ ❖ ❖ / ❖ Stan GAM ❖ ❖ GAM
- 36. TFP TF
- 38. 1. TF: - - :
- 39. 1. TF:
- 40. 1. TF: - - - GPU / TPU Inception v3 Inception v4 # of parameters: 42,679,816 # of layers: 48
- 41. 1. TF: - Keras, Slim - TensorBoard
- 42. 1. TF: - - tf .distributions
- 43. 2. x: x⇤ s P(x | ↵) ✓⇤ ⇠ Beta(✓ | 1, 1)
- 44. 2. - ( ) p(x, ✓) = Beta(✓ | 1, 1) 50Y n=1 Bernoulli(xn | ✓),
- 46. 3. Edward TF
- 47. TFP - TFP = TensorFlow + + + - TensorFlow - - TF GPU, TPU, TensorBoard, Keras - - TensorFlow
- 51. BakFoo, Inc.
- 52. BakFoo, Inc. QUICK: Workstation, Qr1 | AI
- 53. BakFoo, Inc. : SNS +