The Science of Fun - Data-driven Game Development
- 1. The Science of Fun Data-driven Game Development Dr. Alexandra Țurcan, Deep Silver GamesLab Manager Ruan Pearce-Authers, Senior Online Services Engineer
- 2. About us
- 3. Psychologist Researcher Scientist Data scientist in the games industry UX Researcher in Deep Silver GamesLab Alex
- 5. Dambuster Studios We make video games. We are one of Deep Silver’s development studios, along with Volition (USA) and Fishlabs (Germany). You can find us on Canal Street, near Broadmarsh.
- 6. Homefront: The Revolution - is an open-world first person shooter where you must lead the Resistance movement in guerrilla warfare against a superior military force. Homefront "Freedom fighters" trailer
- 8. What is data-driven game development? • Including data in the game development process • This data can be generated from playtests with both internal and external players • It can be used to make more informed decisions during game development • At Dambuster, we collect data in our research lab, Deep Silver GamesLab
- 9. GamesLab: research and playtesting facility
- 10. • Fun is the ultimate goal of the game development process • “Fun” is a subjective experience, and for each player it could come from something else: • easy game, • hard game, • collaborative, • individual, • etc. What is “fun”?
- 11. • Observing player behaviour • Asking for player opinion and feedback UX methods • Used to objectively quantify player in-game behaviour • Take the load off observational methods Telemetry • Used to quantify unconscious player behaviour • Eye tracking, skin conductance Biometrics Million dollar question: How do you measure fun?
- 12. UX
- 13. UX Research • Traditionally, it’s the process of understanding the impact of design (e.g. websites) on an audience • Example methods: • Interviews • Surveys • Observations • Focus groups • Heuristic evaluation / expert review • Adapted to video games, but it doesn’t scale well and it’s not easy to automate • Pro: can answer the “why” questions
- 15. What is telemetry? • Data collected from running games • Used for both development and retail builds • Gives us quantitative behavioural data • Example telemetry: • Discrete events – player started a level, killed an enemy, died, etc • Continuous data streams – player position, health, etc
- 16. Sending telemetry • Game code is instrumented to send events appropriately • As events are emitted, we send them to a remote server • Events are sent in the exact order they’re emitted • This is very useful later on! • Think of this as a stream of data • The first event in a telemetry stream is purely metadata • Platform, user ID, etc
- 17. Telemetry event examples { "type": "telem_start", "t": 1484132366, "v": 1, "data": { "Account": { "User": "UKPC4020", "Platform": "editor" }, "Build": { "Game": "OnlineSample", "Type": "dev" }, "Hardware": { "OS": "windows" } } }
- 18. Telemetry event examples { "type": "player_spawn", "t": 1484132366, "v": 1, "data": { "Name": "PlayerBlueprint_C_0", "Pos": { "X": 326.24, "Y": 3.55, "Z": 268.38 } } }
- 19. Telemetry event bus • A lightweight HTTPS/WebSocket server receives all these events • They’re forwarded into Kafka, a log database • Supports many consumers, with non-destructive reads • We retain data based on storage capacity and lifetimes • Can replay events if we find processing bugs, or need to extract new data
- 20. Telemetry stream processing • We’re now all set to start making use of the data! • Processors listen for one or more specific event types • They have local memory to combine events • Can use RAM directly, or a Redis instance • Output summarised/combined events to: • Separate queues for further processing • Permanent data storage
- 22. Example telemetry processor in JavaScript
- 23. Permanent telemetry storage • We use MongoDB as a default • Rich aggregation framework for queries • Scales well from an ops perspective • But there’s no one-size-fits-all solution to data storage • Special cases include: time-series data • These databases are then used by analysis scripts and dashboards
- 24. R is a functional language and environment for statistical computing and graphics. https://www.r-project.org/ Telemetry analysis
- 27. Telemetry for designers - Where do players die? -
- 28. Telemetry for designers - Can the player see the enemy who’s shooting them? - • Player can’t see the enemy: • not_in_FoV • rendered / not_rendered • Enemy is occluded: • in_FoV • not_rendered • Player could be seeing the enemy: • in_FoV • rendered
- 29. Telemetry for designers - Do players become more accurate shooters over time? - lm(accuracy ~ time) => Model explains 0.004% of variance. Time does not predict accuracy on its own. Other predictors?
- 30. • Player accuracy doesn’t improve over time. • Reasons why that might be: • Not enough practice? • Too long delay between play sessions? • Other variables might be affecting accuracy (e.g. ammo count: when players have loads of ammo they can afford to not be as accurate as when they are running out of ammo)?
- 31. Telemetry for detecting subtle issues
- 32. Telemetry for business intelligence - Timeline of mission popularity -
- 33. Telemetry for online services - What is the relationship between ping and distance? - • Distance is only weakly correlated with session ping (correlation coefficient = 0.28). • With every 1 km increase in distance, the average ping only increases with 0.01 ms. • The value of the ping does not change with distance. • So, a good matchmaking algorithm should not depend solely on geographical distance.
- 35. Why use biometrics at all? • It nicely complements the quantitative behavioural data provided by telemetry and the qualitative player opinions provided by UX methods • It’s the only tool we have that measures unconscious player behaviour • Can’t be faked – player or researcher subjectivity does not affect it
- 36. Eye-tracking • Traditionally used in academic research to study human visual attention • Adopted more and more in the games industry because it can answer questions about player behaviour that cannot be answered through any other means • It is non-invasive – eye-tracking hardware snaps onto the PC monitor and tracks eye movements from a distance - the player will not be inconvenienced in any way
- 37. Skin conductance Galvanic skin response (GSR) aka Skin conductance (SC) aka Electrodermal activity (EDA) • Measures changes in skin electrical conductivity • Reliable indicator of emotional state • Not under conscious control
- 38. Stimulus Activate body’s autonomic nervous system Sweat Skin moisture content increases Skin’s electrical conductivity increases • Even if a person is not visibly sweating, sweat is still building up underneath the skin, which means that GSR can still be measured. How GSR works
- 39. Valence Arousal Can be measured with facial recognition. Can be measured with skin conductance. Emotions
- 41. Eye-tracking technical specs • Works by reflecting near-infrared light off the cornea • The hardware gives us (x,y) screen coordinates • This raw data is entirely useless without game world context • Can’t be included in telemetry directly • Let’s see what can actually be done with this!
- 42. Eye-tracking telemetry (3D) • This tells us which objects in the world are being looked at • Deproject those (x,y) coordinates into a world position and direction • Cast a sphere through the world, see what we hit • Customisable radius for accuracy • When the hit object changes, log a “started_looking” event to telemetry with object details
- 44. Eye-tracking telemetry (2D) • This tells us how the player uses the UI/HUD • The game needs to provide a 2D map of all interesting elements • e.g. Circle for a minimap, small rectangle for a health bar • Perform intersection testing on these elements, log events when this changes
- 45. GSR technical specs • GSR data is recorded in microSiemens • Our sensors operate at 10hz (we get one reading every 100ms) • Roughly speaking: sweat means more current goes through • The data is just a single floating-point value • Suitable for direct inclusion in the telemetry stream • Means we can check response to any in-game events
- 46. Pacing volumes • GSR data can’t be used in absolute terms • Designers add markup objects to the world, called pacing volumes • These define expectations for emotional intensity at given areas in the game • e.g. “calm”, “moderate”, “intense” • We can then check how well our expectations match up with reality
- 47. GSR analysis algorithm • Raw GSR data needs to be analysed before it can be interpreted • An algorithm in R outputs a series of significant peaks coupled with the events that caused them Ledalab algorithm in Matlab (http://www.ledalab.de/) by Benedek & Kaernbach (2010). We translated it to R.
- 49. Example eye-tracking use case
- 50. Example GSR use case
- 51. Other GSR use cases • Can be used to answer questions like: • Are various areas of the game more or less engaging for the player? • Does dying elicit a similar emotional response as wining the mission? • Do players panic when they run out of ammo? • Does removing certain game elements make the game less engaging/more boring?
- 52. • Dark Souls 3 • Homefront: The Revolution GSR overlaid on game-play: GSR tracking Dark Souls 3
- 53. Tools recap
- 54. Get involved! • Sign up for playtests • https://gameslab.dsdambuster.com/ • Contact us @AleTurcan and @returnString