Applying AI in Games (GDC2019)
1 like2,465 views
The document discusses the application of AI in gaming, highlighting its use in enhancing user experiences through functionalities such as better matchmaking, player toxic behavior detection, and automated testing. It showcases case studies like 'Gyakuten Othellonia', explaining how AI-driven recommendations and deep learning are utilized to assist players in deck construction and gaming strategies. Additionally, it emphasizes the advantages of using Google Cloud's machine learning tools for scalability, speed, and performance in game development.
Applying AI in Games (GDC2019)
- 1. Confidential + Proprietary Applying Cloud AI in games with DeNA Samir Hammoudi, Gaming Technical Specialist, Google Cloud Jun Ernesto Okumura, AI System Dept. Project Leader, DeNA Ikki Tanaka, AI System Dept. Data Scientist, DeNA
- 2. Create + Connect + Scale
- 3. Many Applications of AI In Games Player against smarter AI (AI playing like a pro player) Smarter NPC Automated QA testing Replace a human player by a bot when they leave a game Fraud/Cheat detection Player toxicity Break language barrier Recommend the appropriate deck/card/character/equipement IAP recommendations Better matchmaking And so many more!!
- 4. Confidential + Proprietary Atari Go Starcraft Games are an ideal environment for AI Possible Actions 17 361 Millions No. of Moves Per Game 100’s of moves 100’s of moves 1000’s of moves
- 5. 21M Developers <1000’s Deep Learning Researchers <1M Data Scientists Helping Democratize Machine Learning
- 6. Confidential + Proprietary AI building blocks for all developers Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence
- 7. Confidential + Proprietary Google Cloud enables your AI journey Pre-packaged AI solutions Powerful image analysis Vision API Natural chatbot interactions DialogFlow Powerful text analysis Natural Language API Train custom machine learning models AutoML A new middle pathway Custom ML models Support for custom ML Models ML Engine Open source ML TensorFlow Hardware optimised for machine learning Cloud TPUs Create ML models using standard SQL BigQuery ML
- 8. Confidential + Proprietary Why customers choose Google for AI Scale Speed Quality Best performance for AI workloads with customized hardware and Cloud TPUs Instant access to thousands of machines with Google Cloud Pre-trained AI building blocks solve business needs, with the highest quality Accessible Cloud AutoML and ML Engine to customize models for users ranging from citizen data scientists to researchers 1 2 3 4
- 9. Confidential + Proprietary More Advanced? Let’s put some AI in your game! You’re new to ML? Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence Cloud ML Engine Cloud AutoML
- 10. Building Game AI for better user experiences Jun Ernesto Okumura / Ikki Tanaka (DeNA)
- 11. Confidential + Proprietary Jun Ernesto Okumura AI System Dept., AI Team Lead / ML Engineer Speaker Gaming AI team Ikki Tanaka AI System Dept., Data Scientist / ML Engineer Speaker Takeshi Okada AI System Dept., ML Engineer Yu Kono AI System Dept., AI Researcher
- 12. Confidential + Proprietary Gaming in DeNA Gyakuten Othellonia Inhouse Nintendo Alliance Titles ● Super Mario Run ● Fire Emblem Heroes ● Animal Crossing: Pocket Camp Megido 72 Alliance ● Final Fantasy Record Keeper ● Uta Macross 3rd Party ● Granblue Fantasy
- 13. Confidential + Proprietary Problem Setting and Approach
- 14. Strategic app game Based on Board Game (Othello / Reversi) Variety of Characters / Skills Released in 2016 23M downloads so far Region: Japan / Taiwan Gyakuten Othellonia
- 15. Deck and Character Deck: 16 characters (among more than 3,000 options) Character: Each character has its own skills, and status Player Deck Character Detail Status: HP (hit-point) ATK (attack-point) Skill description: When this character can flip just one opponent piece, the attack point will double
- 16. 1. High churn-rate 2. Hard to construct optimal decks / No training field 3. Need better on-boarding support for beginners Issues to be solved
- 17. Our approach Support beginners by two AI functions Deck Recommendation select appropriate characters for deck (Association Analysis) Othellonia Dojo (battle AI) support human-level AI for practicing (Supervised Learning w/ Deep Neural Network)
- 18. Available Data Most recent 9 months, ~billion log entries ● Extract high-skilled player’s deck/battle logs ● Data Augmentation (battle AI) ○ Board rotation, hand piece permutation, … preprocessed logsBigQuery
- 19. Deck Recommendation Released on 6th Nov. 2018 A player select 1. deck archetype, 2. deck cost and 3. favorite character Metrics: Usage, Acceptance rate, Win rate, ... Build my deck!Association Rules Recommendation API Server Deck logs
- 20. Othellonia Dojo (AI Player) Feature released last week A player select 1. deck archetype and 2. AI level Metrics: Usage, Win rate, ... Battle record Deep Neural Network Trained AI model Let’s battle! Battle API Server game status Inference result
- 22. Algorithm of Deck Recommendation Player Inputs Select Leader Select Base Characters Select Well-Associated Characters Recommended Deck
- 23. How Are the Relationships Extracted? ● Association analysis ○ Statistical method to extract the relationships inside large-scale data ○ Relations are evaluated quantitatively with some metrics GOOD RELATION! BAD RELATION… GOOD RELATION!
- 24. Why Did We Choose GCP for Our ML Systems? Powerful and flexible tools for ML and Data Science Scalable and robust infrastructure services App Engine Cloud Functions Cloud Pub/Sub Cloud Datastore Stackdriver Logging Compute Engine Kubernetes Engine BigQuery Cloud Datalab Cloud Storage Cloud Machine Learning
- 25. System Architecture with Google Cloud Platform batch Deck Logs BigQuery Association Analysis Compute Engine Cloud Storage Association Rules Master Data of Game (sqlite) batch Cloud Pub/Sub Cloud Storage Cloud Datastore App Engine Cloud Functions Update Time Association Rules Master Data of Game (pickle) (cron) Recommendation API App Engine User ID Characters Obtained Options of Recommendation Recommended Deck
- 27. 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 Deep Learning Makes AI Like A Human Player 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 Deep Neural Network Action Probability Battle logs of top-tier players Update Network >5000 features >10m battles
- 28. Training Phase AI model trained in the Google Compute Engine is stored in the Cloud Storage training PvP Match Training AI model BigQuery Preprocessing Compute Engine Training Compute Engine Cloud Storage Cloud Storage Battle Record Battle Record AI model AI model
- 29. Inference Phase Deploy the trained AI model stored in the Cloud Storage to CMLE serving Inference API App Engine Scores of Actions AI PvE Match GAE/Python3.7 Auto-scaling Cloud ML Engine Auto-scaling Cloud Storage Inference by AI Cloud Machine Learning Features Scores of Actions Battle Record AI model
- 30. Testing of the Systems High Traffic Estimated RPS (requests per second)RPS Time
- 31. Load / Scalability Testing Auto-scaling of CMLE is very helpful for our case ○ Averaged latency is less than 1 second ○ ~1.3% of requests exceeded our latency limit Requests Errors How to remove this errors? 1:30 PM 2:00 PM 2:39 PM 1:30 PM 2:00 PM 2:39 PM
- 32. Our Approach to Eliminate the Errors We set another backup-endpoint to receive the request that fails to be operated in the main-endpoint Inference API App Engine Inference by AI Cloud Machine Learning Prediction request Successful response Prediction request Inference by AI Cloud Machine Learning main backup 5 sec limit something bad...
- 33. Result of Several Testings No errors at all! Total Error Rates 0.00% 0.50% 1.00% 1.50% No Backup Backup (minNode=10) Backup (minNode=20)
- 34. Confidential + Proprietary Results and Future
- 35. Good adoption from beginners Time#ofuses(beginnerclass) Event to promote deck building Release Deck Recommendation
- 36. Good adoption from beginners Win rate of beginners increased by 5% Time winrateatbattle (beginnerclass) Release Deck Recommendation
- 37. Latency The algorithm is complex, but the latency is low Thanks to GCP, especially auto-scaling, it is running error-free Sat 23 Latency 24ms 22ms 20ms 18ms 16ms Stable latency with most requests Sun 24 Mon 25 Tue 26 Wed 27 Thu 28 Mar 1 Sat 2 Sun 3 Mon 4 Tue 5 Wed 6Fri 22
- 38. Othellonia Dojo (AI Player) Preliminary Results ● Good enough win rate as a practice partner for beginners ● Dojo is used for beginners well ● Latency is stable and fast
- 39. Confidential + Proprietary Best practices Record logs that completely reproduce battles Creating headless game simulator Scalable system components (CMLE, GAE) Project structure & communication
- 40. How You Can Get Started With ML
- 41. Chat translation ● Increasingly a global gaming audience ● Connect users with emotes ● Chat enables deeper conversation ● Language doesn’t have to be a barrier
- 42. Player Toxicity ● People can be horrible to each other ● Bad actors ruin the experience for everyone ● Detecting toxic communications is a manual effort ● Real-time detection
- 43. Simplify Games ● QA & testing ● Player assistance ● Non-player characters
- 44. Confidential + Proprietary Thank you