“Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics,” a Presentation from Sportlogiq
0 likes65 views
The document discusses leveraging vision systems, generative models, and reinforcement learning for sports analytics, particularly in youth sports, to generate reliable and actionable insights from incomplete data. It highlights the challenges of incomplete datasets across various league levels and how generative AI can create complete player data and plausible play sequences. The aim is to develop simplified analytics tools for non-analytical users while ensuring that proactive insights are relevant and valuable for youth sports contexts.
![• Modelling approach is inspired by
language models
• Each event can be equivalent to a
sentence in a document
• A document is an episode of play in
hockey game starting with a faceoff
and ending with a whistle
Example Event Token
Generative AI for Data Creation — Game Events
12
© 2024 Sportlogiq
{"tag": "playerEvent", "period": 2,
"periodTime": 721,
"teamId": 15, "playerId": 1330,
"xAdjCoord": 37.92, "yAdjCoord": 39.48,
"playersOnIce":
[384,380,1321,1330,1753,11581,627,305,1348,1964
,1968,1977],
"teamInPossession": 15, "scoreDifferential": -
1,
"playZone": "oz", "playSection": "eastPoint",
"eventName": "pass", "outcome": "successful"}](https://image.slidesharecdn.com/t1r05javansportlogiq2024-241022131912-10329cc6/85/Using-Vision-Systems-Generative-Models-and-Reinforcement-Learning-for-Sports-Analytics-a-Presentation-from-Sportlogiq-12-320.jpg)
“Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics,” a Presentation from Sportlogiq
- 1. Using Vision Systems, Generative Models and Reinforcement Learning for Sports Analytics Mehrsan Javan Chief Technology Officer Sportlogiq
- 2. • Data Generation: Cost efficient, scalable, and robust data acquisition from videos using fully automated/AI- assisted systems • Insights Generation: Descriptive and predictive analytics and relevant insights for users at scale for performance evaluation, scouting, media content creation, prediction, etc. Sport Analytics Is About Insights Extracted from Data 2 © 2024 Sportlogiq Analytics, Metrics, Insights Game Models Player Tracking Activity Detection
- 3. • Top tier pro leagues (NHL, NFL, EPL) • Accurate raw datapoints from different sources • Advanced analytical skills and deep sports knowledge, inhouse data science teams • 2nd tier pro & draft eligible leagues • Partial and almost accurate raw datapoints • Domain expertise with limited analytical knowledge • 3rd tier leagues and youth games • Partial, incomplete and inaccurate raw data • Limited domain expertise and almost no analytical knowledge Sport Analytics in Different Leagues 3 © 2024 Sportlogiq
- 4. • Analytics foundations are based on uniform and accurate input data • Catered for scouts, players, coaches, GMs, and media • Player/team metrics and physical data • Player archetypes, search and recommendations, scouting tools • Expected value models and reinforcement learning tools for strategy evaluation Sample NHL Player Card in Sportlogiq app Analytics in Professional Leagues 4 © 2024 Sportlogiq
- 5. • Data incompleteness — regardless of how strong our computer visions systems are, we are limited to what the camera sees • Any useful analytical insight requires reliable and uniform input data • Pro level analytics models don’t work out of the box with youth data • Insights relevance — what is relevant in the pro leagues may not be relevant for the youth sports users • Users neither have the sports knowledge nor analytics backgrounds • Educating users takes a long time, in the meantime they need a valuable product Extending Analytics from Pro Leagues to Youth Sports Is Not Straightforward 5 © 2024 Sportlogiq
- 6. • Partial player observations • Player location data exists for a subset of players that are visible in the video • Not necessarily all players are observed and tracked for the whole games Distribution of the fraction of the ice area covered in the camera field of view Data Incompleteness and Inaccuracy — Partial Observations 6 © 2024 Sportlogiq Fraction of the ice surface area visible in each frame Normalized distribution density
- 7. • Partial game events and play sequence data • Players actions and game events have a higher false detection rates compared to pro leagues • Some game events are not collected in youth games Data Incompleteness and Inaccuracy — Partial Observations 7 © 2024 Sportlogiq
- 8. Data Incompleteness and Inaccuracy — Partial Observations 8 © 2024 Sportlogiq
- 9. • Use of Generative AI for data creation • Complete player locations for physical data and metrics generation • Plausible sequences of on puck/ball actions by filling the missing actions and correcting mistakes Generating Complete Data from Partial Observations 9 © 2024 Sportlogiq
- 10. • Input tracking data is a partial set of observed location data extracted using computer vision tracking techniques • Generated output is a complete set of player location data • Auxiliary outputs for instantaneous velocity and acceleration • Player physical metrics • Group behavior modeling to capture team tactics remains a challenge Tracking Data Generation Generative AI for Data Creation — Trajectory Data 10 © 2024 Sportlogiq Partially Observed Location (x, y) Team Roster and Match Metadata Camera Parameters Spatial Attention Networks Complete Location Data (x,y) and Their Derivatives
- 11. • Problem Statement: Given a set of partially accurate and incomplete sequences of on puck/ball actions, generate a plausible play sequence for the whole game • A play sequence is a complete set of events, with the event type, other attributes, outcome, time, location on the ice, and global game state • An abstract example of a play sequence without attributes: {Faceoff, Loose Puck Recover, Pass, Reception, Shot, Blocked Shot, Loose Puck Recovery, Shot, Goal, Whistle} Generative AI for Data Creation — Game Events 11 © 2024 Sportlogiq
- 12. • Modelling approach is inspired by language models • Each event can be equivalent to a sentence in a document • A document is an episode of play in hockey game starting with a faceoff and ending with a whistle Example Event Token Generative AI for Data Creation — Game Events 12 © 2024 Sportlogiq {"tag": "playerEvent", "period": 2, "periodTime": 721, "teamId": 15, "playerId": 1330, "xAdjCoord": 37.92, "yAdjCoord": 39.48, "playersOnIce": [384,380,1321,1330,1753,11581,627,305,1348,1964 ,1968,1977], "teamInPossession": 15, "scoreDifferential": - 1, "playZone": "oz", "playSection": "eastPoint", "eventName": "pass", "outcome": "successful"}
- 13. • Base model: Generative Pretrained Transformer (GPT) which captures the causality in the play sequence data • Over 10 seasons of data from 5 professional hockey leagues were used for training the base model • The model was then tuned with limited number of youth games (a few hundred) • GPT model passed both automated and the human QA systems Game Event Sequence Creation Generative AI for Data Creation — Game Events 13 © 2024 Sportlogiq
- 14. 14 © 2024 Sportlogiq We got the data, but what is the end product?
- 15. • Userbase Segmentation • 10% team and personal coaches • 57% family members who are not parents and are over 56 or mention their grandchild • 33% parents and players • Users are not analytically oriented • 27% have no understanding of stats • 32% understand basic stats • 48% not interested in learning analytics Relevant Insights for the Youth Markets 15 © 2024 Sportlogiq 27% 32% 24% 14% 3% Users' Sport Analytics Knowledge I don’t understand analytics at all I can follow basic stats and analytics discussions I can understand visualizations and chart analytics I can easily analyze and explain chart analytics and data visualizations I examine raw data to create detailed analyses and strategic insights
- 16. • The most powerful and sophisticated tool in sport analytics is Inverse Multi- Agent Reinforcement Learning (MARL) • A game is a sequence of <state, action> with a reward signal, each game state has a quantitative value • All advanced player metrics and profiling are based on the MARL systems • Inverse MARL can tell us what a team/player is trying to optimize What Is Behind Pro Leagues Analytics? 16 © 2024 Sportlogiq
- 17. • A tool already exists that can assign quantitative values to each game state • Reward signal can get modified to measure changes in game states for each team based on offense/defense objectives • Significant changes in the game state values are important moments in the game – highlight-worthy segments • Game/player highlights can get generated automatically from videos Example: value of shots at different locations on ice given a specific game context The Use of Pro Leagues MARL for Youth Games 17 © 2024 Sportlogiq
- 18. • Instead of a generic game highlight, a user specific highlight can get created for each player • Using a variety of reward signals, based on a player position (offense, defense) a highlight reel of offensive or defensive plays can get created • Highlights are accompanied with light stats • Highlighted segments are marked based on how interesting they are Highlight Example User Specific Highlights 18 © 2024 Sportlogiq
- 19. • End users in youth sports want simple yet valuable insights • Pro level insights won’t be useful in the youth sports in their current shape and form • Producing reliable sport analytics content requires vast amount of uniform and accurate data • Generative AI can fill the gaps in the input data • Complete player location data can get generated from partial observations • A plausible play sequence for on puck action can get generated from inaccurate and partial play sequence data • MARL can be used to generate player specific highlights • Sport data is multimodal and requires multi-modal generative AI tools Youth Sports Product Development — Conclusions 19 © 2024 Sportlogiq
- 20. For More Information M. Horton, Learning feature representation from football tracking, Sloan sport analytics 2021 US Patents and pending applications 11,130,040; 18/529,204; 17/445,354; 17/817,454 20 © 2024 Sportlogiq Videos All videos and demos are available at https://sportlogiq.com References Liu et al., Learning agent representation for ice hockey, NeurIPS2020