Developing Social Games in the Cloud
22 likes3,249 views
The document discusses the development of social games through cloud computing, specifically the use of Amazon Web Services (AWS) and their various offerings such as IaaS and PaaS. It details the infrastructure needed for running games, including server management, configuration, monitoring, and scaling strategies, while addressing the benefits and challenges of using AWS. Additionally, it outlines how Gatcha integrates with game developers to enhance social gaming experiences and provides insight into the financial aspects of using cloud services.
Developing Social Games in the Cloud
- 1. Developing Social Games in the Cloud Pieter De Schepper Jurriaan Persyn
- 2. Tags Cloud Computing IaaS (Infrastructure as a Service) PaaS (Platform as a Service) AWS (Amazon Web Services) Amazon EC2 Social Applications Multi Platform Game Development
- 3. Who we are Pieter De Schepper Jurriaan Persyn previously working on Netlog previously part of the front-end APIs, OpenSocial & other development team focusing on integrations in Netlog user features Netlog team member for 3 Netlog team member for 4 years years
- 4. Gatcha! wants to bring people together through gaming wherever they are.
- 5. Gatcha! builds and distributes games to social networks and web-portals.
- 6. Who we are For Game Developers For Site Owners We offer: We offer: • Reach & distribution • Quality gaming content • Social Enrichment • Social engagement • Features • Monetization • High-scores • Achievements • Challenges • ... • Monetization • Localization
- 7. Every single day: over 1M game plays over 2200 days of play time
- 8. Where we come from Grown and successful as one of Netlogʼs features Since Jan 2010 a dedicated team of 8 with a more stand-alone approach to go cross-platform.
- 10. UC Berkeley RAD Lab Definition Huge Resources The illusion of infinite computing resources available on demand, thereby eliminating the need for users to plan far ahead for provisioning resources. No Commitment The elimination of an up front commitment by users, thereby allowing companies to start small and increase resources only when there is an increase in their needs. Pay By The Drink The ability to pay for use of computing resources on a short-term basis as needed (e.g., processors by hour and storage by the day) and release them as needed.
- 11. Types Infrastructure as a Service (IaaS) Amazon EC2, GoGrid, Eucalyptus, ... Platform as a Service (PaaS) Google AppEngine, Windows Azure, Amazon Simple DB, ... Software as a Service (SaaS) Gmail, Google Documents, ...
- 12. Why IaaS? You donʼt want to worry ... • about building an expensive datacenter • about the location of your datacenter • about finding the right ITS team & field engineers • about broken discs • about hardware getting out of fashion • ... (minimal CAPEX, Capital Expenditures)
- 13. But? We have a dedicated team We have a datacenter We have negotiated contracts So, why?
- 14. Our main goal ... • instant scaling • the promise of hardware on demand • handling peaks in traffic • handling heavy single tasks • cost optimization
- 15. Why Amazon? Simple Storage Service (S3) since March 2006 Elastic Cloud Computing (EC2) since August 2006 Almost immediately de facto standard for cloud computing Others: Rackspace, GoGrid
- 16. Amazon Web Services IaaS S3: Simple Storage Service EC2: Elastic Compute Cloud EBS: Elastic Block Storage PaaS SQS: Simple Queue System RDS: Relational Database Service SDB: SimpleDB Cloudfront: Content Delivery
- 17. What is Amazon EC2? Amazon EC2 is a virtual computer grid provisioning virtual environments on Amazon infrastructure
- 18. What is Amazon EC2? (contʼd) Elastic capacity easy scaling within minutes Full Control root-level system access Flexible multiple instance types & sizes Can be used in combination with other services e.g. S3, SimpleDB, SQS, ... Secure network security model
- 19. Amazon Concepts AMI - Amazon Machine Images packaged-up environment for setting up and booting your instance Availability Region geographically dispersed in separate geographic areas or countries Availability Zone distinct datacenter locations that are engineered to be insulated from failures External IPʼs mapped via NAT on Amazon routers 1 external IP -> 1 internal IP (max 2 IPs / instance) Security firewall security groups
- 20. Amazon Concepts (contʼd) CU - Computing Unit 1 CU = 1 GHz 2007 opteron/xeon processor each instance has a certain number of CUʼs IAM - Identity Authentication Management AWS Credentials & X.509 Certificates Load Balancing AWS auto-scaling API Web API, Command-Line Tools Management AWS Management Console Firefox Elasticfox plug-in Security firewall security groups
- 21. Storage Solutions EC2 Instance Default Local Storage Ephemeral virtual disks that are integral part of EC2 VM instance Range from 160GB to 1.7 TB total space Only use this for temporary storage
- 22. Storage Solutions (contʼd) EBS - Elastic Block Share Persistent disk volumes that can be mounted on your instances 1 GB to 1 TB per volume, default quota of 20 volumes You can create any filesystem on EBS volumes Easy to backup with snapshots Usage: • Create volume (same availability region) • Attach to instance • Create filesystem • Mount filesystem in your operating system
- 23. Storage Solutions (contʼd) S3 File Storage Reliable, Web URL accessible file-based storage Doesnʼt really store files, but objects, buckets and keys Max 5GB per bucket, unlimited amount of buckets Use: • Backups • Storing Bundled AMIʼs
- 24. </hype>
- 25. <?php // Getting Started // with Amazon EC2 echo "Hello Cloud!"; ?>
- 27. Amazon EC2 sign up Contact information Billing information credit card Pricing information since August 2006
- 29. The first Instance Choosing an AMI from a list of Amazon AMI user uploaded AMIʼs (be careful of which AMIʼs you trust) typical LAMP setups available Gearman AMIʼs, memcached AMIʼs, MySQL AMIʼs, ... Creating a key-pair = security credentials public/private keypair for authentication Creating a security group = firewall rules Access instance ssh -i keypair.pem root@ec2- <...>.compute-1.amazonaws.com
- 30. More tools RESTful / SOAP API several php API clients available some maintained by Amazon Command Line Tools launch/terminate/... from CLI ElasticFox FF plugin similar to AWS console
- 31. Gatcha EC2 Setup Management Servers Application servers Inventory Management Web Servers Configuration Management Databases Monitoring Service Memcache Servers DNS Service Gearman Load Balancing Service Smartfox ... Development server
- 32. Gatcha EC2 Setup (contʼd) Inventory Management Which servers are running? What is their role? What is their instance id? Read: SQLite database and API with list of instances and their core meta-data
- 33. Gatcha EC2 Setup (contʼd) Configuration Management via puppet: “Open Source Configuration Management Tool” Custom declarative language to describe system configuration “Templates” to describe parts of the system users / services / packages On deploy Puppet puts managed systems into declared state Read: Puppet installs right version of Apache, MySQL. Configures users, access, ...
- 34. Gatcha EC2 Setup (contʼd) Monitoring via Zabbix: “Open Source Distributed Monitoring Solution” (Amazon offers “CloudWatch”) Read: Periodic polling of all instances and get health & performance parameters. Trigger alerts when needed.
- 35. Gatcha EC2 Setup (contʼd) Internal DNS Server via Bind9: “Berkeley Internet Name Domain DNS Server” Read: Instances talk to each other via hosts names. hostname to ip translation via internal DNS server
- 36. Gatcha EC2 Setup (contʼd) Load Balancing via Amazon Elastic Load Balancing = Amazon Service Amazon provides failover Public via Elastic IP Distributing incoming traffic between all instances in same availability zone Performs health checks (polling of instances) Also possible via own instances, eg. HaProxy AMIʼs Read: Responsible for distributing user requests to array of web servers
- 37. Gatcha EC2 Setup (contʼd) Applications Servers Servers that need CPU & Servers that need Memory persistent storage web servers databases gearman order services: Amazon SDB, Amazon other services: Amazon SQS RDS smartfox memcached other services: gear6 2 AMIʼs: instance store AMI EBS AMI
- 38. (Auto) Scaling Typical set up of new web server custom AMI w/ Debian distribution boot script sets up SSH keys configures hostname updates internal DNS server further provisioning via Puppet ultimately health checks to Zabbix contact AmazonLoadBalancer API and start serving requests
- 39. (Auto) Scaling Zabbix sees high load on web servers? Triggers bash script to create new web launch new instance on Amazon add to Inventory Management boot scripts and puppet bring it into production Zabbix sees low load on web servers? Triggers bash script to terminate instance
- 40. Backups Backups via: “Open Source Network Backup Solution” Read: Separate service that polls Inventory Management for list of database. Create EBS snapshots for volumes of those DBs
- 41. ... but ...
- 42. Gotchaʼs
- 43. Gotchaʼs Default storage = instance storage (ephemeral) If instance goes down -> storage is gone Persistent storage: use EBS bases AMIʼs No external static IP available for instances Use Elastic IP (only when necessary) No guarantee that your instance will run forever No need to worry about hardware, but... what is Amazon doing, how, when, why? Donʼt trust any AMI youʼll stumble upon
- 44. Some obstacles Database server crashes Eventually found out it was related to kernel. No possibility to create own kernels. The most recent kernel on Amazon was 3 years old Experience with Amazon Support wasnʼt that smooth.
- 45. Some obstacles (contʼd) Custom VPN Still working out the best way to create and manage the VPN over both Netlog & Gatcha servers. Amazon servers are a maze of networks. Difficult to control in custom VPN. Amazon offers Virtual Private Cloud Currently only in US-EAST
- 46. Extra services RightScale “Cloud Computing Management Platform” A management tool, with different features atop of AWS, enabling you to take full control of all concepts of your clouds Also provides support and consultancy You pay for what you get (pretty expensive) Power users: • Zynga (FarmVille, Maffia wars, Zynga Poker, ...) • Playfish (Hotel City, Word Challenge, ...)
- 47. Extra services (contʼd) Scalr Cheaper alternative to RightScale Part Open Source Focus Easy scaling: scaling your database, application servers and load balancers Monitoring: automatically replaces servers when one fails Backups: makes automated backups to Amazon EBS
- 48. Impact on application Focus on application needs, not on hardware specs. What do I need today? Upgrade hardware when itʼs needed. (e.g.. start with normal instance for you db, upgrade to large, extra- large when needed.)
- 49. Impact on application (contʼd) Dynamic Configuration e.g.. New Memcached server? Add it to your configuration files (Use consistent hashing algorithms for Memcached)
- 50. Impact on application (contʼd) Stand alone system Try to identify components in your application that could stand alone. Loosely-coupled systems are easier to scale.
- 51. Impact on application (contʼd) Auto Scaling not simple for database servers Adding MySQL slaves can be automated for scaling READS Scaling databases is still the hardest part AWS only helps us in scaling infrastructure. Scalability is more than that.
- 52. Impact on application (contʼd) “Everything fails, all the time.” Avoid single points of failure. Design for failure.
- 53. Conclusion Set-up Initial set-up is very simple and easy. Enough solutions, partners and documentation to help you out. For custom stuff, it gets harder. Integration with existing monitoring / backup / configuration management not that simple. You depend on Amazon. Thereʼs always the unexpected stuff.
- 54. Conclusion The money? + No expensive set-up costs. No long term commitments to make. No planning for hardware. Impact of performance improvements (cpu/ bandwidth) has direct relation to money saved. - Consultancy and Cloud manager tools are expensive.
- 55. Conclusion Pricing Scenario per month Set up: 10 web & memcached servers 2 gearman job processors 2 databases 1TB provisioned storage 3TB data transfer from public ips (these are not the Gatcha! figures)
- 56. Conclusion Amazon Elastic Compute Cloud - EU (Ireland) Region Amazon EC2 running Linux/UNIX $0.38 per Large Instance (m1.large) instance-hour (or partial hour) e.g.. 10 web, memcached, load balancer, ... servers 10 * 24 * 31 = 7400 hours $2812 $0.76 per Extra Large Instance (m1.xlarge) instance-hour (or partial hour) e.g.. 2 database servers 2 * 24 * 31 = 1488 hours $1130 $0.76 per High-CPU Extra Large Instance (c1.xlarge) instance-hour (or partial hour) e.g.. 2 gearman worker server 2 * 24 * 31 = 1488 hours $1130
- 57. Conclusion Amazon EC2 EBS $0.11 per GB-month of provisioned storage e.g.. 1TB-Mo $110 $0.11 per 1 million I/O requests e.g.. 2OOM IOs $22 $0.15 per GB-Month of snapshot data stored e.g.. 10GB-Mo $15 $0.010 per 10,000 gets (when loading a snapshot) e.g.. 150k requests $0.15 $0.010 per 1,000 puts (when saving a snapshot) e.g.. 30k requests $0.30
- 58. Conclusion Elastic IP Addresses $0.01 per non-attached Elastic IP address per complete hour e.g.. 1 ip you want to keep in use 1 * 24 * 31 = 744 hours $8 Elastic Load Balancing $0.028 per LoadBalancer-hour (or partial hour) e.g.. 1 load balancer 1 * 24 * 31 = 744 hours $21 $0.008 per GB Data Processed by the LoadBalancer insignificant
- 59. Conclusion AWS Data Transfer (excluding Amazon CloudFront) (EU (Ireland) Region) $0.150 per GB - up to 10 TB / month data transfer out e.g.. 200GB $30 $0.000 per GB - first 1 GB / month data transfer out $0 $0.000 per GB - data transfer in (free Until June 30, 2010) $0 $0.010 per GB - regional data transfer - in/out/between EC2 Avail Zones or when using public/elastic IP addresses or ELB e.g.. 3TG $30
- 60. Total: $5308
- 61. Conclusion The money? (contʼd) If you have a fairly static set up, own hardware will be cheaper. Not being able to quickly scale has itʼs price too. Ideally: use hybrid set up where Amazon helps in handling peak traffic or unexpected traffic. (Typical situation: 1/3 of servers only in use for 1/5 of the time, to be able to handle peak.)
- 62. Conclusion Get: Flexible resource handling Minimal capital expenses Need: Efficiency needs automation Effectiveness needs experience
- 65. Conclusion AWS allows you to focus on product and users Amount of hardware available buys you time. You can temporarily solve problems with “throwing hardware at the problem”
- 66. VELO PERS FO R DE
- 67. A ME High-score Handling R G YOU Tournaments A ME Challenge builder IA LG Achievements S OC OP AT Got an idea for a game? Great!
- 68. Gatcha For Game Developers Game tracking Start game and end game calls results in accurate gameplay tracking and allows us to show who is playing the game at any given moment, compute popularity, target games. High-scores You push your high-score to our API, we do the hard work of creating different types of leader boards and rankings. Achievements Pushing achievements reached in your game, just takes one API call, no configuration needed.
- 69. Gatcha For Game Developers Multiplayer Games We run SmartFox servers that enable you to build real-time multiplayer games, with e.g.. in game chat coming: Challenges & Tournaments Allow your game players to challenge each other, or build challenges & contests yourself.
- 70. Gatcha For Game Developers How to integrate? Flash Games We offer wrapper for AS3 and AS2 games with full implementation of our API Unity3D Games OpenSocial Games Talk to the supported containers via the Gatcha OpenSocial Extension Other Games Simple iframe implementation. PHP Client API available for the Gatcha API Start developing in our sandbox.
- 74. Job openings Weʼre searching for great developers! PHP Talents Working on integrations and the gaming platform Flash Developers Working on Flash Games and the gaming platform Design Artists Designing games and integrations
- 75. Want to know more? Ask. pieter@netlog.com jurriaan@netlog.com toon@netlog.com www.gatcha.com
- 76. sources developer.amazonwebservices.com hyperstratus.com rightscale.com eucalyptus.com jochen-lillich.de gatcha.com pictures Iko - flickr.com/photos/iko/106957481 Marco Fulle - antwrp.gsfc.nasa.gov