Entertainment Architectures 2011
1 like377 views
The document discusses the integration of IBM into the interactive entertainment industry, highlighting its role in the architecture and infrastructure of massively multiplayer online games and social media platforms. It outlines the technical achievements and management strategies of George Dolbier, including building scalable systems capable of supporting millions of users and driving significant revenue growth. Additionally, it provides insights into entertainment as a service, emphasizing architectural patterns and infrastructure necessary for creating and delivering interactive content.
Entertainment Architectures 2011
- 1. Architectures in Entertainment George Dolbier Session: Z99 IBM Day/Date/Time
- 2. Today’s Agenda IBM, Games? Really? And who is this guy? Interactive Entertainment Industry Architectural patterns for Scalable Entertainment Internals Massively Multiplayer Online Games
- 3. Who is IBM again? And who is this guy?
- 4. IBM is embedded the gaming industry starting with the CPUs in every game console Souce www.nexgenwars.com 2/27/2011
- 5. To the largest Online Infrastructures: EvE Online Active Servers Weight CPUS RAM Estimated FLOPS ~ 195 About 2.5 Metric Tonnes > 420 CPU cores > 1 THZ > 7.5 Terabytes > 7.5 Terra Total Bandwidth Estimate ~ 400 Gbs
- 6. George Dolbier, IBM CTO Interactive Media 20+ year Technological innovator and Business Developer successfully pushing the limits of possibilities daily. A classically trained computer scientist with product production experience ranging from embedded software development to database kernel engineer at Informix and Oracle and operating system kernel engineer at Sequent. Management and business experience ranges from IT operation and tactical management of small task oriented teams to executive level positions managing large geographically dispersed groups. Relevant Experience Architecture/design/development/deployment/operations of 30+ Interactive Media projects including social netowrks, streaming video, Online Games, Cinema in 15+ countries 20+ years in SW Technology, 15 years in Interactive Media Architect and software designer with deep experience in producing software products and web scale services in the, social media, MMO, Film and connected console game Successful projects in 15+ countries in the 3 major Geographies of Asia North America and Europe Developer/Publisher relations include: CCP, Lucas, EA, Unity, Epic, Sony, Norsk Tipping, Obsidian, Zynga, Monumental, Cartoon Network, Hasbro Technology Reltationships include BigWorld, Unity, Instant Action (FKA Garage Games) Vivox, Sleepy Giant, Parature, Epic Executive experience Manage, drive and mentor teams up to up to 1500 Manage geographically dispersed teams Produce consistent sustainable double digit Y2Y profit and revenue growth Led creation of 10 million dollar Joint venture between IBM, Intel and Cisco Led creation of relationships between IBM and Games industry providers (BigWorld,Vivox,GNi) Drove business unit to build global revenue from 0 to 180+ million in under 4 years Technical Delivery experience Deployed 6000 servers infrastructure in china to support 3 million players in under 6 weeks Doubled Epic’s production environment allowing project to be developed on time on budget Implemented production environment at ILM Used on StarTrek,TransFormers2, Harry Potter Architected & Implemented HPC Cluster that discovered genetic cause of bone marrow cancer Education Key Skills • IBM Executive Architect Certified • Open Group Certified Master Architect • Portland State University – BS Computer Science • Online Technology Architecture • Online Operations • Global Rollout • Social Media Architecture • IM Production • DFX Production • Follow the Sun dev. mgmt. • Strategy & Operational • Production Optimization • Technical Capacity Analysis • OT&B Delivery
- 7. Industry Introduction Convergent Divergencies A little History It’s How big?
- 8. Interactive Entertainment is Convergence Online Technical • Persistence • Centralized Simulation • Scale • CC Pipeline Business • Software As a Service • Re-accruing revenue • Subscription • Economic Systems/Item Sales • Ongoing Operational Expense Interactivity • Scale • Autonomy • Co-Ordination • Co-Operation • Arbitration • Economics • Communication Community • Communications • In Game Support • Access to the game from outside Interactive Entertainment Is… A scalable, persistent, form of entertainment delivered as a service
- 9. It’s How Big? • EvE Online: 340,000 Subscribers • Bejeweled Blitz: 10 Million Monthly active Users • World of Warcraft: 12 Million Monthly Subscribers • Lineage II: 19 Million Players world wide • FrontierVille: 19 Million Monthly Active Users • Farmville: 85 Million Monthly active users • Cityville: 100 Million monthly active users http://www.eveonline.com/pressreleases/ http://us.blizzard.com/en-us/company/press/pressreleases.html?101007 http://pc.ign.com/articles/925/925099p1.html http://www.appdata.com/leaderboard/apps (2/27/2011)
- 10. The Facebook Top 40 Top 40 FB Apps: Monthly Active 0 20,000,000 40,000,000 60,000,000 80,000,000 100,000,000 CityVille Texas Phrases Conduit QuizTaco! Yahoo! PetSociety Yelp MindJolt FriendBuzz Zoosk Birthday Games MallWorld MAU Top 40 FB pps: MonthlyActive (minus farmville & Cityville) 0 10,000,000 20,000,000 30,000,000 40,000,000 50,000,000 MAU All Over: 2 Million MAU
- 12. © 2008 IBM Corporation Every Architecture exists in an Environment SettingIDCStudio InteractionOperationProductionCreation Segment Creation: Content is created Production: Content is transformed Operation: Content is distributed and operated Interaction: Content is interacted with Time Concept Launch Flight Technology deployment timeline Environments Studio: Highly Collaborative Environment where concepts are created and transformed into products. This is where the virtual world is created IDC: Consolidated, efficient, secure Environment where data and communications come together. This is where the virtual world runs Setting: Home, Office, Mobile, Capability, device, location, and connectivity that allows a game or virtual world to be interacted with Environments are differentiated by economic model as well as physical, technical and Human factors Disciplines are differentiated by the goals of individuals, and how those individuals relate to content
- 13. © 2008 IBM Corporation Entertainment Creation: The Studio Segment Creation: Content is created Production: Content is transformed Operation: Content is distributed and operated Interaction: Content is interacted with Time Concept Launch Flight Technology deployment timeline Environments Studio: Highly Collaborative Environment where concepts are created and transformed into products. This is where the virtual world is created IDC: Consolidated, efficient, secure Environment where data and communications come together. This is where the virtual world runs Setting: Home, Office, Mobile, Capability, devic e, location, and connectivity that allows a game or virtual world to be interacted with Environments are differentiated by economic model as well as physical, technical and Human factors Disciplines are differentiated by the goals of individuals, and how those individuals relate to content Setting Interaction IDC Operation Studio ProductionCreation
- 14. © 2008 IBM Corporation Content Creation Architecture
- 15. © 2008 IBM Corporation Entertainment as a Service: The Internet Datacenter Segment Creation: Content is created Production: Content is transformed Operation: Content is distributed and operated Interaction: Content is interacted with Time Concept Launch Flight Technology deployment timeline Environments Studio: Highly Collaborative Environment where concepts are created and transformed into products. This is where the virtual world is created IDC: Consolidated, efficient, secure Environment where data and communications come together. This is where the virtual world runs Setting: Home, Office, Mobile, Capability, device, location, and connectivity that allows a game or virtual world to be interacted with Environments are differentiated by economic model as well as physical, technical and Human factors Disciplines are differentiated by the goals of individuals, and how those individuals relate to content Studio ProductionCreation Setting Interaction IDC Operation Some might call this the service deliver environment… (we shun them)
- 17. Why one v.s. the other Sharded Content Multiple Operations Authored Content Step Scale Moderate latency Universe Content Single Operations User created Content Scale Out Latency Tolerant Hybrid Content Mixed Operations Instanced Content Bi-Directional Scale Low Latency
- 18. © 2008 IBM Corporation Interactive Content Delivery This is NOT streaming video…. This is NOT A transactional workload
- 19. Internals: Massively Multiplayer Online Games Architecture for millions of players
- 20. More than just the game engine
- 22. Economic Services Financial administration Interface Game Interface User Financial Account Database Account Management Website Tax/VAT Tax processing system Support Administration Interface FireWall DOS/DDOS Account Transaction Database Decision Support Database Fraud Detection/ Reporting System Financial Transaction processing Payment Processor Gateways
- 23. Support Services FireWall CSR Interface Support Interface to Financial System In Game Support Interface Problem Management database Decision Support Database External Support Website Support Ticketing System CRM system Support Knowledge Base (Wiki) Interface to bug tracking system Interface to game/ Systems & Network Management DOS/DDOS
- 24. Game Services Support Administration Interface Financial administration Interface Player behavior reporting system Map/Object Data Mnaagement and comms log Player Stats Game Website Simulation Engine FireWall DOS/DDOS Routing/load balencing “Instance” Manager Authentication System Patch Server CDN Authentication System Txt Chat Server Voice Chat Server Simulation Engine Simulation Engine Simulation Engine “Instance” Manager XBLA/PSN GW Transaction management/ caching Transaction management caching In Game Economic system Console Network Game Manager Game Manager Game Manager Game Manager Simulation Engine Simulation Engine Simulation Engine Simulation Engine Txt Chat Log
- 25. CCP’s EVE online Specific examples of the importance of DB performance in online entertainment These next slides where originally presented by Derek Wyse, CCP VP Ops, at the EvE Fanfest 2011
- 26. Our Goals • Keep up with load • Redundancy at every level • Simplification of design • Options!
- 27. Redundancy
- 28. Simplification • Devices • Old TQ = 8 devices • Proposed Design = 10 devices • Final Design = 6 devices • Entire system can go live as a unit... No partial upgrade need • System maintainable on all units live (using the standby path)
- 29. OMGWTF DATA • Step one: (Upgrade the Storage Network) – WIN: 8Gb/s to 32Gb/s and no sharing ;) • Step two: (Upgrade the RAM) – WIN: 128GB of DDR2 to 512GB of DDR3, 111GB of DB Cache to 460 DB Cache and it all takes our Page Life Expectancy (the time data stays in RAM) from 7.9 mins to 2.8 hours! Also, the old TQ database was at 97% Buffer Cache Hit Ratio while the New TQ database is at 99.7%. • Step three: (Upgrade the CPU) – WIN: 12 to 32 logical processors plus a bonus for the new architecture gain over the old processor in our test and in lab tests. Plus, seriously... that screen shot by its self makes the server 400% faster. • Step four: (Upgrade the Storage) – WIN: Transfer speed from 2.9Gb/s to 10Gb/s tested. Total storage capacity on SSD's doubled. Aggregate storage on all production capable tiers increased from 2TB non redundant to 11.5TB and 1 for 1 redundant hardware.
- 32. OMGWTF DATA • Each server has: – 2 x 8 Core Intel x7560 processors at 2.26GHz – 32 x 16GB strips of RAM for a total of 512GB DDR3 RAM – 4 x Gigabit Ethernet cards – 2 x Dual-port 8Gb/s Fiberchannel cards – 4 x 15K 300GB SAS drives internal – 9 x 300 GB SSD's drives RAID 5 attached via IBM V7000 – 36 x 600 GB SAS drives RAID 10 attached via IBM V7000 • The whole system has: – Over 1TB of RAM – 64 logical processors (w/ hyper-threading enabled) – 32Gb/s of storage throughput capacity – 200,000 IOPS capacity – 51TB of RAW storage capacity (23TB after RAID and spares)
- 33. OMGWTF DATA • Backup – Server Disk System Time Speed – Old TQ DB Current (30) FC Disks 73 mins 2.8 Gb/s – New TQ DB Current (30) FC Disks 60 mins 3.2 Gb/s – New TQ DB New (34) SAS Disks 23 mins 8.5 Gb/s • Restore – Server Disk System Time – Old TQ DB Current (30) FC Disks 180 mins – New TQ DB Current (30) FC Disks 66 mins – New TQ DB New (34) SAS Disks 59 mins
- 34. Internals: Social Games A couple orders of magnitude larger How to support HUNDREDS of millions
- 35. Special Notes: Social Games • Farmville had 1 Million daily players after 4 days and 10Million after 60 days. • 75 million monthly users in 9 months (from 0 to >France) • Most “games” are 80-90% Read, primarily non- transactional workload • Social Games heavy write load: Farmville 30% • Social game load spikes can be > 50% http://highscalability.com/blog/2010/2/8/how-farmville-scales-to-harvest-75-million-players-a-month.html
- 36. Architecture stack for Social Game Data Persistance Layer DataCenter Sesson Mgmt WebServices DB Cache Physical server DB DB DB Physical server DB DB DB Physical server DB DB DB DB Cache DB Cache DB Cache DB Cache GameSerivces PHP Game Services GameSerivces PHP GameSerivces PHP Web Layer Sesson Mgmt WebServices Sesson Mgmt WebServices Cache Cache Cache Cache Cache Sesson Mgmt WebServices Physical server DB DB DB Comm Services router Load balencer firewall Load balencer Load balencer router firewall
- 37. Questions ?!? 4/27/2013Template Presentation - Session Z99 37
- 38. Thank You
- 39. Architectures in Entertainment George Dolbier georged@us.ibm.com