Anticipation Mappings for Learning Classifier Systems
3 likes599 views
The document discusses anticipatory classifier systems (ACS) and their ability to predict the effects of actions in a given state. It presents a method for enhancing learning systems to compute anticipations and update classifier parameters based on previous state-action pairs. The approach is claimed to provide accurate predictions with smaller populations and is generalized to handle real-valued and noisy domains.
![Anticipatory Prediction Predicting the next state Classifiers matching s t that advocate a t For each action a in [M], the classification accuracy C(x t ,a) is computed as, Learning to predict the next state Use current state s t , a t , and s t+1 to train the vector w](https://image.slidesharecdn.com/anticipation-mappings-for-learning-classifier-systems2107/85/Anticipation-Mappings-for-Learning-Classifier-Systems-8-320.jpg)
Anticipation Mappings for Learning Classifier Systems
- 1. Anticipation Mappings for Learning Classifier Systems Larry Bull, Pier Luca Lanzi, Toby O’hara University of the West of England, Bristol, UK Politecnico di Milano, Italy Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana Champaign, USA CEC 2007, September 27th, 2007, Singapore TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A A A A A A
- 2. A Brief Look at Classifier Systems If condition C holds in state S , then action A will produce a payoff p , this prediction has an accuracy F
- 3. Anticipatory Classifier Systems (ACS) Modify the structure of classifiers and the systems to learn the effect of actions Short history Rick L. Riolo. 1990 “Lookahead Planning and Latent Learning in a Classifier System.” Wolfgang Stolzmann. 1998 “Anticipatory Classier Systems”, GP98 Martin V. Butz (2002), “Anticipatory Classifier Systems”, Springer-Verlag If condition C holds in state S t , then action A will produce an effect resulting in state S t+1
- 4. Need Anticipations? Compute them! Basic Idea Not a system designed for anticipations Enrich a known system to learn anticipations Anticipatory prediction comes for free Maybe less powerful than ACS, but can be applied to any LCS If condition C holds in state S , then action A will produce a payoff p , with an accuracy F If condition C holds in state S , then action A will produce a payoff p , with an accuracy F , and effect is an f (s t , w) First tried by Larry Bull and Toby O’hara (2000) using neural networks to compute an f (s t ,w)
- 5. Learning Anticipatory Functions s t s t+1 a t s t a t -> s t+1
- 6. (our way to) Anticipations Classifiers has five parameters: The condition and the action The prediction, the error and the fitness The parameter vector w for anticipation If condition C holds in state S , then action A will produce a payoff p , with an accuracy F , and effect is an f (s t , w)
- 7. Learning to Anticipate the Effect The next state s t+1 is used to update w of classifiers Classifier error is updated according to a is the action computed by the classifier ε f (x t ,y t ,a) is the error function Several error functions, we used the simplest one: 0 if action is correct (a = y t ), 1000 otherwise Classifier fitness is updated as in XCS
- 8. Anticipatory Prediction Predicting the next state Classifiers matching s t that advocate a t For each action a in [M], the classification accuracy C(x t ,a) is computed as, Learning to predict the next state Use current state s t , a t , and s t+1 to train the vector w
- 9. What Function? Sigmoid Action is simply a constant: Update is performed with a gradient descent: a f xw
- 10. What Function? Neural Networks When the problem involves many action we can either use an array of simple action function use a powerful action function Neural Network n inputs h hidden nodes As many outputs as the action encoding needs Update is performed using online backpropagation
- 11. Woods
- 12. Simple Sequential Problems (1) (2) (3)
- 13. Anticipation Accuracy (1) accuracy MSE
- 14. Anticipation Accuracy (2) accuracy MSE
- 15. Anticipation Accuracy (3) accuracy MSE
- 16. Alias
- 17. Sequential Problems with Aliasing (1) (2) Extension of anticipatory Classifier Systems for problem with noise was developed by Martin Butz, Dave G. Goldberg, and Wolfgang Stolzmann (2000)
- 18. Anticipation Accuracy (1) accuracy MSE
- 19. Anticipation Accuracy (2) accuracy MSE
- 20. We presented a very simple approach to anticipatory behavior Compute the anticipation of the next state based on the previous state and the action performed Very simple, but provide accurate predictions, while requiring smaller populations Simpler than ACSs, probably less powerful Even simple perceptron can be powerful enough Generalizes to real-valued and/or noisy domains
- 21. Any Question? Thank you!
- 22. Woods1 The Woods1 model is very sparse : there are only 128X17 state-action pairs Woods1 is, in practice, more simple than binary sum Both perceptron and NN does not exploit the problem sparseness as SVM
- 23. Maze 5 Maze5 is still a sparse problem but slightly more complex: it has 288X37 state-action pairs Maze5 is, in practice, slight more difficult than binary sum Also in this case SVM exploits very effectively the problem sparseness