Ai for games seminar: N-Grams prediction + intro to bayes inference
Download as PPTX, PDF5 likes4,416 views
The document discusses the use of n-gram prediction and Bayesian inference techniques for action prediction in games, highlighting their applications in gaming AI. It covers how player behavior can be analyzed to enhance adaptive AI, making gameplay more dynamic and less predictable. The document also outlines decision learning methods and their implications for developing intelligent game characters.
![ACTION PREDICTION
N-Grams
● Record the probabilities for each move given all
combination of choices for the previous “N”
moves.[Millington]
● “N” = window size + 1
○ 3-Gram keeps track the probabilities for a window
size of 2
● Update the list when new actions are performed
● Predict the one with higher probability
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux](https://image.slidesharecdn.com/aiforgamesseminarn-gramandbayes-141027154935-conversion-gate01/85/Ai-for-games-seminar-N-Grams-prediction-intro-to-bayes-inference-9-320.jpg)
![ACTION PREDICTION
N-Grams - Window Size
Accuracy of N-Grams for the Left-Right game.[Millington]
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux](https://image.slidesharecdn.com/aiforgamesseminarn-gramandbayes-141027154935-conversion-gate01/85/Ai-for-games-seminar-N-Grams-prediction-intro-to-bayes-inference-13-320.jpg)
![DECISION LEARNING
Naive Bayes Classifiers - Example
Shoot? Distance Direction Markers
Yes Medium Central Low
Yes Large Central None
No Short Central High
No Large Angular High
Yes Short Angular None
No Medium Angular High
Yes Medium Central Medium
No Short Central High
Yes Medium Angular Low
[...] [...] [...] [...]
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux](https://image.slidesharecdn.com/aiforgamesseminarn-gramandbayes-141027154935-conversion-gate01/85/Ai-for-games-seminar-N-Grams-prediction-intro-to-bayes-inference-19-320.jpg)
![DECISION LEARNING
Literature and Further Readings
● Artificial Intelligence for Games [Ian Millington, John Funge]
● AI Games Programming Wisdom [Steve Rabin]
● Teaching Bayesian behaviours to video game characters [Ronan Le
Hy, et al]
● MIT open courseware
● PR-OWL
● Bayesian Networks [Judea Pearl, Stuart Russell]
● Design and Development of a Compound DSS for Laboratory
Research [Tomáš Hujer]
● Bayesian Artificial Intelligence [Kevin B. Korb, Ann E. Nicholson]
● Player Prediction techniques for AI in video games [Michael
Webersdorfer]
● Adaptive AI for fighting games [Ricciardi, Thill
N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux](https://image.slidesharecdn.com/aiforgamesseminarn-gramandbayes-141027154935-conversion-gate01/85/Ai-for-games-seminar-N-Grams-prediction-intro-to-bayes-inference-29-320.jpg)
Ai for games seminar: N-Grams prediction + intro to bayes inference
- 1. AI for Games Seminar by Andrea Tucci N-GRAM PREDICTION AND BAYES INFERENCE 27/10/2014 Slides by Andrea Tucci - @andreatux
- 2. OUTLINE ➔ Context ➔ Action Prediction ◆ Techniques ◆ N-Grams ● Window Size ◆ Considerations ➔ Decision Learning OUTLINE ◆ Techniques ◆ Naive Bayes Classifiers ◆ Bayes Networks ● Bayes Inference ● Probability Propagation ● Games Application N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 3. OUTLINE OUTLINE - 2 ➔ Conclusion ➔ Literature and Further Readings ➔ Questions N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 4. OUTLINE ● Learning in games Context ○ more dynamism, basing on the player ○ online and offline ○ build a bot from user data/decision ○ difficult to debug Uses ● Parameter Modification ● Action Prediction ● Decisions ● ...more N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 5. ACTION PREDICTION Action Prediction ● Study the player’s behaviour, guess next move ● Adaptive AI: the player cannot use the same technique ● Same challenge, different experience ● A little bit of randomness is needed… ○ otherwise the player will predict the bot! N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 6. ACTION PREDICTION ● Probability Techniques ○ store data of the player’s choices ○ pick the most predictable + very easy to implement - the player can soon learn the mechanism ● String Matching ○ a string is used to store the player’s decisions ○ to predict the next move, the string is parsed ○ search for identical situation in the string ○ prediction is based on history of choices N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 7. ACTION PREDICTION Example: Techniques - 2 ● In a RPG context the player can perform a physical attack and 3 spells: Fire, Blizzard, Thunder. ○ Encoding: ■ Physical Attack: “A” ■ Fire : “F” ■ Blizzard: “B” ■ Thunder “T” ● During a battle we store his/her decisions ○ “AAFBAFBAATTFBBAAFB” N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 8. ACTION PREDICTION ● Next move? Techniques - 3 ● Search in the string what the player did in the past, after performing a Fire spell followed by a Blizzard spell. ● AAFBAFBAATTFBBAAFB (?) ○ A physical attack is more predictable ● Window size : how many moves should be looked up ○ is 2 in the example above ○ important for the performance of the prediction N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 9. ACTION PREDICTION N-Grams ● Record the probabilities for each move given all combination of choices for the previous “N” moves.[Millington] ● “N” = window size + 1 ○ 3-Gram keeps track the probabilities for a window size of 2 ● Update the list when new actions are performed ● Predict the one with higher probability N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 10. ACTION PREDICTION N-Grams - 2 Example: Left or Right? LRRLLLRLLRLRL..(?) 3-Gram predictor ->R ->L LL 2/3 1/3 LR 1/4 3/4 RL 1/3 2/3 RR 0/1 1/1 N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 11. ACTION PREDICTION N-Grams - 3 Easy to predict things like... N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 12. ACTION PREDICTION N-Grams - Window Size ● Determine the accuracy of the prediction ● Increasing the window size means better prediction ○ ...until it get worse ● Optimal window size should not be too large ○ Long sequences tend to have too much randomness N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 13. ACTION PREDICTION N-Grams - Window Size Accuracy of N-Grams for the Left-Right game.[Millington] N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 14. ACTION PREDICTION Considerations ● The more the window increase in size, the more memory is required to store probabilities ● Best performance are achieved when sequences are “known” ○ otherwise learning takes a while… ● Very suitable for combat games, computational linguistics, computational biology and in general pattern-based actions N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 15. DECISION LEARNING Decision Learning ● Learn how to make decisions ● From a set of possible actions, AI learn which one apply basing on observation data ● Weak and strong supervision ○ give feedback to the AI choices ● Huge amount of information in games N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 16. DECISION LEARNING Techniques ● Decision Tree Learning ● Artificial Neural Networks ● Naive Bayes Classifiers ● Reinforcement Learning N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 17. DECISION LEARNING Naive Bayes Classifiers ● Decide an action from previous observation ○ can be other players decisions, in the same context ● Basing on evidence, gives the probability of “how good” is executing an action ● Bayes rule N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 18. DECISION LEARNING Naive Bayes Classifiers - 2 ● Core concept: express the conditional probability of A given B, in terms of its inverse, B given A. ○ the probability that it rained can be expressed as the observation that the ground is wet Example: in a soccer game, should we shoot at goal? ● Base our decision on a training set ● Try to find decision patterns ● Data in the example is labeled ○ this help to generalise similar context N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 19. DECISION LEARNING Naive Bayes Classifiers - Example Shoot? Distance Direction Markers Yes Medium Central Low Yes Large Central None No Short Central High No Large Angular High Yes Short Angular None No Medium Angular High Yes Medium Central Medium No Short Central High Yes Medium Angular Low [...] [...] [...] [...] N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 20. DECISION LEARNING Naive Bayes Classifiers - Example ctd. Should we shoot if... ○ distance = medium ○ direction = angular ○ markers = none P ( shoot? | dist, dir, mark) = P(shoot? | dist) * P(shoot? | dir) * P(shoot? | mark) ● Applying the bayes rule, we have an indication based on the data collected ○ Google Docs ● Player should shoot! N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 21. DECISION LEARNING Bayes Networks ● Reason under uncertainty ○ AI reasons in a human like way, no cheats ● Arrange variables and their relationship in a “belief network” ○ acyclic graph ○ variables are encoded as node ○ relationships are arcs ○ variable probability distribution encoded in a table ● Inference using probabilistic reasoning ○ Causal Inference: given A → B, we know A ○ Diagnostic Inference: given A → B, we know B ○ Intercausal Inference: given A → C and B → C, we know C N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 22. DECISION LEARNING Bayes Networks - 2 Away from home, our neighborhood tell us that he heard our alarm rang.. What happened? ● P(A) = P(B) * P(E) * P(A|B,E) N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 23. DECISION LEARNING Bayes Inference P(B) P(E) P(A|B,E) P(A) N. P(A) T=.001 T=.002 .95 .000002 .000795 T=.001 F=.998 .94 .000938 .372814 F=.999 T=.002 .29 .000579 .230127 F=.999 F=.998 0.001 .000997 .396264 .002516 1 ● If the news inform us on an earthquake… the probability that a burglary occurred as well, are low! ○ B → A, E → A. P(E) changed and P(B) changed as well N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 24. DECISION LEARNING Bayes Inference - 2 Determining the probability of a burglary after the news P(B) P(E) P(A|B,E) P(A) N. P(A) T=.001 T=1 .95 .00095 .003269 T=.001 F=0 .94 0 0 F=.999 T=1 .29 .28971 .996731 F=.999 F=0 .001 0 0 .29066 1 N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 25. DECISION LEARNING Bayes Inference - 3 ● Find the probability distribution over a variable X given the evidence e ○ P(X | e) ? ● Inference can be exact or approximated ● Computing the probability of a node: summing the probabilities of the parent nodes ○ chain rule N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 26. DECISION LEARNING Probability Propagation ● The belief of each node is updated by prior or posterior evidence updates N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 27. DECISION LEARNING ● Visibility Bayes Networks in ○ “fog of war” ● Dependency graphs games ○ RTS technology graph ■ infer existence of some technologies by the presence of others ■ which dependencies a player is attaining ■ etc ● Detect Intruder N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 28. DECISION LEARNING Conclusion ● Learning can be used for many reasons in game development ● Main question.. do we need it? ○ balance effort of building a learning system with outcomes ● Offline learning is often preferred ● Learning a whole new behaviour is not so easy ● Adaptive AI can be the future of gaming ○ ideally no repetitive challenges N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 29. DECISION LEARNING Literature and Further Readings ● Artificial Intelligence for Games [Ian Millington, John Funge] ● AI Games Programming Wisdom [Steve Rabin] ● Teaching Bayesian behaviours to video game characters [Ronan Le Hy, et al] ● MIT open courseware ● PR-OWL ● Bayesian Networks [Judea Pearl, Stuart Russell] ● Design and Development of a Compound DSS for Laboratory Research [Tomáš Hujer] ● Bayesian Artificial Intelligence [Kevin B. Korb, Ann E. Nicholson] ● Player Prediction techniques for AI in video games [Michael Webersdorfer] ● Adaptive AI for fighting games [Ricciardi, Thill N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux
- 30. CONCLUSION That’s it! Thank you for your attention Questions? N-gram Prediction and Bayes Inference • Andrea Tucci - @andreatux