You are not alone - Scaling multiplayer games
8 likes2,808 views
The document discusses the challenges and considerations in scaling multiplayer games to accommodate over a million daily users, highlighting issues related to server databases and client-server interactions. It mentions the need for effective synchronization, lag compensation, and efficient distribution without centralization to enhance gameplay experience. Additionally, it emphasizes learning from past game development efforts and encourages a collaborative approach among independent teams.
You are not alone - Scaling multiplayer games
- 1. You are not alone Scaling Multiplayer Games Jesper Richter-Reichhelm - @jrirei Knut Nesheim - @knutin
- 9. > 1,000,000 daily users
- 10. > 1,000,000 daily users > 50,000 concurrent users
- 11. > 1,000,000 daily users > 50,000 concurrent users > 10,000 requests / second
- 12. Ok, you can handle that.
- 13. Ok, you can handle that. But why only single player?
- 14. The Games The Past The Present The Future The Foundation
- 15. Server Database
- 16. Server Database
- 17. Server Database
- 18. Server Database
- 19. Server Database
- 20. Server Database
- 21. DB is the bottleneck
- 23. Server Database
- 24. Server Database
- 25. Server Database
- 26. Server Database One Game Session
- 27. Server Database One Game Session
- 28. session
- 29. session session session session
- 30. Server session session session session
- 31. Server session session session session
- 32. Server session session session session
- 33. Server session session session session
- 34. Server Server Server session session session session session session session session session session session session
- 35. The Games The Past The Present The Future The Foundation
- 36. es Thoughts t Client ent Server re Operation ation
- 37. es Thoughts t Client ent Server re Operation ation
- 39. You are not alone
- 40. Pessimistic
- 43. Pessimistic
- 44. Client Server A A Client Server B B
- 45. Client Server A @ B A Client Server B B
- 46. Client Server A @ B A Client Server B B
- 47. Client Server A @ B A Client Server B B
- 48. Client Server A @ B A Client Server B B’
- 49. Client Server A @ B A Client Server B B’
- 50. Client Server A @ B A Client Server B B’
- 51. Client Server A @ B’ A Client Server B B’
- 52. Client Server A @ B’ A Client Server B B’
- 53. Client Server A @ B’ A Client Server B’ B’
- 54. Waiting on server responses?
- 55. Waiting on server responses? Are you out of your mind?
- 56. Schizophrenic
- 57. Client Server A A Client Server B B
- 58. Client Server A @ B A Client Server B B
- 59. Client Server A @ B A + B Client Server B B
- 60. Client Server A @ B’ A + B Client Server B B
- 61. Client Server A @ B’ A + B Client Server B B
- 62. Client Server A @ B’ A + B’ Client Server B B
- 63. Client Server A @ B’ A + B’ Client Server B B
- 64. Client Server A @ B’ A + B’ Client Server B B
- 65. Client Server A @ B’ A + B’ Client Server B’ B
- 66. Client Server A @ B’ A + B’ Client Server B’ B
- 67. Client Server A @ B’ A + B’ Client Server B’ B’
- 68. What about real interactions?
- 69. What about real interactions? And what about complexity in the backend?
- 70. Optimistic
- 71. Client Server A A Client Server B B
- 72. Client Server A @ B A Client Server B B
- 73. Client Server A @ B’ A Client Server B B
- 74. Client Server A @ B’ A Client Server B B
- 75. Client Server A @ B’ A Client Server B B
- 76. Client Server A @ B’ A Client Server B B’
- 77. Client Server A @ B’ A Client Server B B’
- 78. Client Server A @ B’ A Client Server B B’
- 79. Client Server A @ B’ A Client Server B’’ B’
- 80. Client Server A @ B’ A Client Server B’’ B’
- 81. Client Server A @ B’ A Client Server B’’ B’
- 82. Client Server A @ B’ A Client Server B’ B’
- 84. Hmm, client rollbacks? Are you sure you can build that?
- 86. es Thoughts t Client ent Server re Operation ation
- 87. Client Server A A Client Server B B
- 90. Client A
- 91. state count 0 max 3 Client A
- 92. user A move state count 0 max 3 Client A
- 93. user A move pick state count 0 max 3 Client A
- 94. user A move pick state count 1 max 3 Client A
- 95. user A move pick state count 1 max 3 Client A Server
- 96. user A user A move pick state count 1 max 3 Client A Server
- 97. user A user A move pick loca,on state count 0 count 1 max 3 Client A Server
- 98. user A user A move pick loca,on state count 1 count 1 max 3 Client A Server
- 99. user A user A move pick loca,on state count 1 count 1 max 3 Client A Server
- 100. user A user A move pick loca,on state count 1 count 1 max 3 history move A pick A Client A Server
- 101. user A user A move pick loca,on state count 1 count 1 max 3 history move A pick A Client A Server
- 102. user A user A move pick loca,on state count 1 count 1 max 3 history move A pick A Client A Server
- 103. user A user A move pick loca,on state count 1 count 1 max 3 user B history move A pick A Client A Server
- 104. user A user A move pick loca,on state count 1 count 1 max 3 user B history move A pick A loca,on Client A Server
- 105. user A user A move pick loca,on state count 2 count 1 max 3 user B history move A pick A loca,on Client A Server
- 106. user A user A move pick loca,on state count 2 count 1 max 3 user B history move A pick A loca,on Client A Server
- 107. user A user A move pick loca,on state count 2 count 1 max 3 user B history move A pick A pick B loca,on Client A Server
- 108. user A user A move pick loca,on state count 2 count 1 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 109. user A user A move pick loca,on state count 2 count 2 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 110. user A user A move pick loca,on state count 2 count 2 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 111. user A user A move pick pick loca,on state count 2 count 2 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 112. user A user A move pick pick loca,on state count 2 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 113. user A user A move pick pick loca,on state count 2 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 114. user A user A move pick pick loca,on state count 2 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 115. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 116. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 117. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick A pick B loca,on Client A Server
- 118. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick A pick B loca,on pick B Client A Server
- 119. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick pick A pick B loca,on pick B Client A Server
- 120. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick pick A pick B loca,on pick B Client A Server
- 121. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick pick A pick B loca,on pick B Client A Server
- 122. user A user A move pick pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick pick A pick B loca,on pick B Client A Server
- 123. user A user A loca,on state count 3 count 0 max 3 user B user B history move A pick A pick B loca,on pick B Client A Server
- 124. user A user A move loca,on state count 3 count 0 max 3 user B user B history move A pick A pick B loca,on pick B Client A Server
- 125. user A user A move pick loca,on state count 3 count 1 max 3 user B user B history move A pick A pick B loca,on pick B Client A Server
- 126. user A user A move pick loca,on state count 3 count 2 max 3 user B user B history pick move A pick A pick B loca,on pick B Client A Server
- 127. user A user A move pick loca,on state count 3 count 3 max 3 user B user B history pick move A pick pick A pick B loca,on pick B Client A Server
- 128. ... (Let’s look inside)
- 129. es t ent re ation
- 130. es Thoughts t Client ent Server re Operation ation
- 131. Erlang
- 133. Erlang Process isolation Networking
- 134. Erlang Process isolation Networking Distribution
- 135. User process
- 136. User process Stores user state
- 137. User process Stores user state Real-time push to client
- 138. World process
- 140. World process Serializes updates Concurrency “control”
- 141. World process Serializes updates Concurrency “control” Runs game logic
- 142. Game logic
- 143. Game logic Pure
- 144. Game logic Pure f(State, Cmd) -> {NewState, Reply}
- 145. Game logic Pure f(State, Cmd) -> {NewState, Reply} Transactions
- 146. Real-time push
- 148. Real-time push Browser support “Transfer-Encoding: chunked”
- 149. Real-time push Browser support “Transfer-Encoding: chunked” State updates as JSON
- 150. es t ent re ation
- 151. es Thoughts t Client ent Server re Operation ation
- 152. Elli
- 154. Elli Efficient, robust High-throughput, low latency
- 155. Elli Efficient, robust High-throughput, low latency Request-response model
- 156. Elli Efficient, robust High-throughput, low latency Request-response model github.com/knutin/elli
- 157. Distribution
- 158. Distribution Idea: Identical nodes, Riak
- 159. Distribution Idea: Identical nodes, Riak No centralization/serialization
- 160. Distribution Idea: Identical nodes, Riak No centralization/serialization No SPOF
- 161. Distribution needs
- 162. Distribution needs Locking
- 163. Distribution needs Locking Dynamic cluster
- 164. Meet “locker”
- 165. Meet “locker” Distributed lock table
- 166. Meet “locker” Distributed lock table CAS
- 167. Meet “locker” Distributed lock table CAS Write quorum, 2PC
- 168. Meet “locker” Distributed lock table CAS Write quorum, 2PC Asynchronous replication
- 169. Meet “locker” Distributed lock table CAS Write quorum, 2PC Asynchronous replication 330 SLOC, one gen_server
- 170. The Games The Past The Present The Future The Foundation
- 171. Research
- 174. Everything is possible First person
- 175. Everything is possible First person Real-time strategy
- 176. Everything is possible First person Real-time strategy Turn-based
- 177. Everything is possible First person Real-time strategy Turn-based Etc
- 178. Nothing is possible
- 179. Nothing is possible Network latency kills gameplay
- 180. Lag compensation
- 181. Lag compensation Many solutions
- 182. Lag compensation Many solutions Depends on game
- 183. FPS
- 184. FPS Speed is king
- 185. FPS Speed is king Divergent simulations
- 186. FPS Speed is king Divergent simulations Server keeps history
- 187. FPS Speed is king Divergent simulations Server keeps history Validates delayed commands
- 188. RTS
- 190. RTS Synchronization is king Identical simulations
- 191. RTS Synchronization is king Identical simulations Server buffers commands
- 192. RTS Synchronization is king Identical simulations Server buffers commands Broadcasts, lockstep
- 193. The Games The Past The Present The Future The Foundation
- 195. Each team adds something
- 196. Based on previous learnings
- 197. Standing on the shoulders of giants
- 200. Questions?