Effects of change propagation resulting from adaptive preprocessing in multicomponent predictive systems
0 likes296 views
The document discusses the importance of adaptive preprocessing in multicomponent predictive systems (MCPS) to manage changes in data streams. It highlights the need for reactive adaptation strategies to ensure system consistency and prevent crashes caused by evolving data. The authors propose extending the definition of MCPS to include change propagation using a new token for meta-data.
![Need of preprocessing adaptation
Streaming error (mean over last 10 samples)
SYN dataset with GFMM classifier
GFMMZ-Score PCA Min-Max
Wrongly classified
(out of [0,1])
New hyperboxes](https://image.slidesharecdn.com/kes2016presentation-160907124349/85/Effects-of-change-propagation-resulting-from-adaptive-preprocessing-in-multicomponent-predictive-systems-18-320.jpg)
![Reactive adaptation of MCPS
GFMMZ-Score PCA Min-Max
Time
i p1
p2
p3
o
[-3.1, 2.7]
x1
= 3.6](https://image.slidesharecdn.com/kes2016presentation-160907124349/85/Effects-of-change-propagation-resulting-from-adaptive-preprocessing-in-multicomponent-predictive-systems-23-320.jpg)
![Reactive adaptation of MCPS
GFMMZ-Score PCA Min-Max
Time
i p1
p2
p3
o
data
meta-data
[-3.1, 2.7]
x1
= 3.6
[-3.1, 3.6]](https://image.slidesharecdn.com/kes2016presentation-160907124349/85/Effects-of-change-propagation-resulting-from-adaptive-preprocessing-in-multicomponent-predictive-systems-24-320.jpg)
![Reactive adaptation of MCPS
GFMMZ-Score PCA Min-Max
Time
i p1
p2
p3
o
data
meta-data
prediction
[-3.1, 2.7]
x1
= 3.6
[-3.1, 3.6]](https://image.slidesharecdn.com/kes2016presentation-160907124349/85/Effects-of-change-propagation-resulting-from-adaptive-preprocessing-in-multicomponent-predictive-systems-25-320.jpg)
Effects of change propagation resulting from adaptive preprocessing in multicomponent predictive systems
- 1. Effects of change propagation resulting from adaptive preprocessing in multicomponent predictive systems Manuel Martín Salvador, Marcin Budka, Bogdan Gabrys {msalvador,mbudka,bgabrys}@bournemouth.ac.uk Data Science Institute. Bournemouth University KES-2016, York, UK September 7th, 2016
- 2. Outline 1. Prologue 2. Introduction to MCPS 3. Motivation 4. Reactive adaptation of MCPS 5. Experiments 6. Conclusion
- 3. PROLOGUE
- 4. Butterfly effect Small causes can have large effects — Edward Lorenz (1917 - 2008) Source: GloWings
- 5. Change propagation Controlled change management in a system CC by TheGiantVermin
- 6. Data streams “Infinite” number of records Continuously arriving to the system at different or same rates Can be stationary or evolving
- 7. Data streams Examples: ● Sensors in manufacturing industry ● Traffic monitoring sensors ● Event logs in websites ● Transactions in the financial sector “Infinite” number of records Continuously arriving to the system at different or same rates Can be stationary or evolving A single engine of Airbus A320 has more than 1000 sensors generating 10GB/s!!
- 9. Data Stream Data stream learning for online prediction Predictive Model Online Supervised Learning Algorithm Predictions True labels t+k t
- 10. Data Stream Data stream learning for online prediction Predictive Model PredictionsPreprocessing Postprocessing Multicomponent Predictive System (MCPS)
- 11. MCPS composition Manual ● WEKA ● RapidMiner ● Knime ● IBM SPSS Automatic ● Auto-WEKA (Bayesian optimisation) ● Auto-sklearn (Bayesian optimisation + Meta-learning) ● TPOT (Genetic programming) ● e-Lico IDA (Ontologies + Planning) Example of WEKA workflow
- 12. Formalising MCPS o token (data) i place transition Well-handled and Acyclic Workflow Petri net (WA-WF-net) MCPS = (P, T, F)
- 13. Formalising MCPS o prediction i place transition Well-handled and Acyclic Workflow Petri net (WA-WF-net) MCPS = (P, T, F) “Automatic composition and optimisation of multicomponent predictive systems” @ IEEE TNNLS (under review) http://bit.ly/automatic-mcps-tnnls
- 14. Formalising MCPS Classifier o Replace missing values Dimensionality reduction Outlier handling token (data) i place transition Well-handled and Acyclic Workflow Petri net (WA-WF-net) MCPS = (P, T, F) “Automatic composition and optimisation of multicomponent predictive systems” @ IEEE TNNLS (under review) http://bit.ly/automatic-mcps-tnnls
- 15. MOTIVATION
- 16. Data changes over time Snapshot of SYN dataset at different times
- 17. Need of model adaptation Streaming error (mean over last 10 samples) SYN dataset with GFMM classifier GFMMZ-Score PCA Min-Max Wrongly classified
- 18. Need of preprocessing adaptation Streaming error (mean over last 10 samples) SYN dataset with GFMM classifier GFMMZ-Score PCA Min-Max Wrongly classified (out of [0,1]) New hyperboxes
- 19. Main strategies for MCPS adaptation Adaptation strategies GLOBAL LOCAL Re-composition Full Partial Hyperparameter optimisation (keep components) Full Partial Parameterisation (keep components and hyperparameters) Full Partial
- 20. Main strategies for MCPS adaptation Adaptation strategies GLOBAL LOCAL Re-composition Full Partial Hyperparameter optimisation (keep components) Full Partial Parameterisation (keep components and hyperparameters) Full Partial “Adapting Multicomponent Predictive Systems using Hybrid Adaptation Strategies with Auto-WEKA in Process Industry” @ AutoML / ICML 2016 http://bit.ly/adapting-mcps-paper This work!
- 21. Need of change propagation Streaming error (mean over last 10 samples) SYN dataset with GFMM classifier GFMMZ-Score PCA Min-Max Inconsistent hyperboxes due to a different input space
- 22. REACTIVE ADAPTATION OF MCPS
- 23. Reactive adaptation of MCPS GFMMZ-Score PCA Min-Max Time i p1 p2 p3 o [-3.1, 2.7] x1 = 3.6
- 24. Reactive adaptation of MCPS GFMMZ-Score PCA Min-Max Time i p1 p2 p3 o data meta-data [-3.1, 2.7] x1 = 3.6 [-3.1, 3.6]
- 25. Reactive adaptation of MCPS GFMMZ-Score PCA Min-Max Time i p1 p2 p3 o data meta-data prediction [-3.1, 2.7] x1 = 3.6 [-3.1, 3.6]
- 26. Updating a component: GFMM 0 1 1 0 (-3.1) (2.7) x1 x2 0 1 1 0 (-3.1) (3.6) x1 x2 Hyperboxes are mapped to the new input space
- 27. EXPERIMENTS
- 28. Experiments Name # Attr # Class Type SYN 2 2 Synthetic ELEC 7 2 Real COVERTYPE 54 7 Real GAS 128 6 Real Datasets Scenarios Id Adap. Model Adap. Prepro. Change Propagation #1 No No No #2 Yes No No #3 Yes Yes No #4 Yes Yes Yes First 200 samples for initial training, rest 400 for testing and online learning GFMMZ-Score PCA Min-Max
- 29. Results #3 crashes due to lack of change propagation when changing PCA components
- 30. CONCLUSION
- 31. Conclusion Only model adaptation may not be enough to cope with evolving data streams, adaptive preprocessing should be considered. However, “blind” adaptation of components can result in inconsistent models or even in a system crash. Local adaptation of a component may require adapting further components. Therefore, a system must be reactive and propagate changes. The definition of MCPS has been extended to support change propagation using a new token for meta-data in a coloured Petri net (cMCPS).
- 32. Future work Large study to measure the actual cost of adaptation. Open questions: ● How to handle propagation requiring changes of the Petri net structure? ● How to handle transformations in systems with nonlinear components? ● How to order components to reduce the cost of adaptation? ● Can a meta-data token be removed at an early stage instead of being fully propagated?
- 33. Thanks! Paper: http://bit.ly/change-propagation-mcps Slides: http://www.slideshare.net/draxus Manuel <msalvador@bournemouth.ac.uk> @draxus