![Tribal Crossing: Representing Game Data in Couchbase
Player Object
Key: 'Player1' Plant Object
Array Key: 'Plant201'
(
[Id] => 1 Array
[Name] => Shawn (
) [Id] => 201
[Player_Id] => 1
[Name] => Starflower
PlayerPlant List )
Key: 'Player1_PlantList'
Array
(
[0] => 201
[1] => 202
[2] => 204
)
31](https://image.slidesharecdn.com/couchbase-bjmeetup-110708042519-phpapp01/85/Couchbase-b-jmeetup-31-320.jpg)
![Tribal Crossing: Accessing Game Data in Couchbase
Get all plants belong to a given player
Request: GET /player/1/farm
$plant_ids = couchbase->get('Player1_PlantList');
$response = array();
foreach ($plant_ids as $plant_id)
{
$plant = couchbase->get('Plant' . $plant_id);
$response[] = $plant;
}
echo json_encode($response);
33](https://image.slidesharecdn.com/couchbase-bjmeetup-110708042519-phpapp01/85/Couchbase-b-jmeetup-33-320.jpg)
![Tribal Crossing: Modifying Game Data in Couchbase
Give a player a new plant
// Create the new plant
$new_plant = array (
'id' => 100,
'name' => 'Mushroom'
);
$couchbase->set('Plant100', $new_plant);
// Update the player plant list
$plant_ids = $couchbase->get('Player1_PlantList');
$plant_ids[] = $new_plant['id'];
$couchbase->set('Player1_PlantList', $plant_ids);
34](https://image.slidesharecdn.com/couchbase-bjmeetup-110708042519-phpapp01/85/Couchbase-b-jmeetup-34-320.jpg)
Couchbase b jmeetup
- 1. USING COUCHBASE FOR SOCIAL GAME SCALING AND SPEED Steven Mih, Couchbase Inc. Bin Cui, Couchbase Inc. 1
- 2. Agenda • Introduction • What is Couchbase Server? – Simple, Fast, Elastic – Technology Overview (Architecture, data flow, rebalancing) • Tribal Crossing Inc: Animal Party – Challenges before Couchbase • Original Architecture – Why Couchbase? • Simplicity • Performance • Flexibility – Deploying Couchbase • New Architecture • EC2 • Data Model • Accessing data in Couchbase • Product Roadmap • Q&A 2
- 3. Couchbase Inc. • Membase and CouchOne have merged to form Couchbase Inc. (headquartered in Silicon Valley) • Team – Brings together the creators and core contributors of Memcached, Membase and CouchDB technologies – Doubles technical team size, accelerates roadmaps by over a year • Products – Couchbase Server (Formerly Membase) – Couchbase Single Server – Mobile Couchbase (iPhone and Android) • Technology – Most mature, reliable and widely deployed NoSQL technologies – Fully featured, open source document datastore – First complete, end-to-end NoSQL database product 3
- 4. Modern Interactive Web Application Architecture Application Scales Out Just add more commodity web servers www.facebook.com/animalparty Load Balancer Web Servers Relational Database Scales Up Get a bigger, more complex server Database - Expensive and disruptive sharding - Doesn’t perform at Web Scale 4
- 5. Couchbase Server is a distributed database Couchbase Web Console Application user Web application server Couchbase Servers 5
- 6. Couchbase data layer scales like application logic tier Data layer now scales with linear cost and constant performance. Application Scales Out www.facebook.com/animalparty Just add more commodity web servers Load Balancer Web Servers Couchbase Servers Database Scales Out Just add more commodity data servers Horizontally scalable, schema-less, auto-sharding, high-performance at Web Scale Scaling out flattens the cost and performance curves. 6
- 7. Couchbase Demonstration • Membase ServerTemplate Demo – Starting with four database Application user nodes under load – Dynamically scaling to eight database nodes Web application server – Easy management and monitoring – Not possible any other Couchbase Servers database technology In the EC2 or Datacenter 7
- 8. Couchbase Server is Simple, Fast, Elastic • Five minutes or less to a working cluster – Downloads for Windows, Linux and OSX – Start with a single node – One button press joins nodes to a cluster • Easy to develop against – Just SET and GET – no schema required – Drop it in. 10,000+ existing applications already “speak Couchbase” (via memcached) – Practically every language and application framework is supported, out of the box • Easy to manage – One-click failover and cluster rebalancing – Graphical and programmatic interfaces – Configurable alerting 8
- 9. Couchbase Server is Simple, Fast, Elastic • Predictable – “Never keep an application waiting” – Quasi-deterministic latency and throughput • Low latency – Built-in Memcached technology – Auto-migration of hot data to lowest latency storage technology (RAM, SSD, Disk) – Selectable write behavior – asynchronous, synchronous (on replication, persistence) • High throughput – Multi-threaded – Low lock contention – Asynchronous wherever possible – Automatic write de-duplication 9
- 10. Couchbase Server is Simple, Fast, Elastic • Zero-downtime elasticity Couchbase Web Console – Spread I/O and data across commodity servers (or VMs) – Consistent performance with linear cost – Dynamic rebalancing of a live cluster • All nodes are created equal – No special case nodes – Clone to grow • Extensible – Change feeds – Real-time map-reduce – RESTful interface for management 10
- 11. Proven at Small, and Extra Large Scale • Leading cloud service (PAAS) • Social game leader – FarmVille, provider Mafia Wars, Empires and Allies, • Over 150,000 hosted applications Café World, FishVille • Couchbase Server serving over • Over 230 million monthly users 6,200 Heroku customers • Couchbase Server is the primary database behind key Zynga properties 11
- 12. Customers and Partners Customers (partial listing) Partners 12
- 13. Couchbase Server Architecture 11211 11210 memcapable 1.0 memcapable 2.0 moxi REST management API/Web UI vBucket state and replication manager Global singleton supervisor Rebalance orchestrator Configuration manager memcached Node health monitor Process monitor protocol listener/sender Heartbeat Data Manager Cluster Manager engine interface Couchbase Storage Engine http on each node one per cluster Erlang/OTP HTTP erlang port mapper distributed erlang 8091 4369 21100 – 21199 13
- 14. Couchbase Server Architecture 11211 11210 memcapable 1.0 memcapable 2.0 moxi vBucket state and replication manager REST management API/Web UI Global singleton supervisor Rebalance orchestrator Configuration manager memcached Node health monitor Process monitor protocol listener/sender Heartbeat engine interface Couchbase Storage Engine http on each node one per cluster Erlang/OTP HTTP erlang port mapper distributed erlang 8091 4369 21100 – 21199 14
- 15. Couchbase “write” Data Flow – application view User action results in the need to change the VALUE of KEY 1 Application updates key’s VALUE, 2 performs SET operation 4 Couchbase client hashes KEY, identifies 3 KEY’s master server SET request sent over network to master server 5 Couchbase replicates KEY-VALUE pair, caches it in memory and stores it to disk 15
- 16. Couchbase Data Flow – under the hood SET request arrives at SET acknowledgement KEY’s master server 1 3 returned to application 2 2 Listener-Sender Listener-Sender Listener-Sender RAM* RAM* RAM* membase storage engine membase storage engine Couchbase storage engine 2 SSD SSD SSD SSD SSD SSD SSD SSD SSD Disk Disk Disk 4 Disk Disk Disk Disk Disk Disk Replica Server 1 for KEY Master server for KEY Replica Server 2 for KEY 16
- 17. Elasticity - Rebalancing Node 1 Node 2 Node 3 Before vBucket 1 vBucket 7 • Adding Node 3 vBucket 2 vBucket 8 vBucket 3 vBucket 9 • Node 3 is in pending state vBucket 4 vBucket 10 Pending state • Clients talk to Node 1,2 only vBucket 5 vBucket 11 vBucket 6 vBucket 12 vBucket 1 vBucket 7 During vBucket 2 vBucket 8 vBucket 3 vBucket 9 • Rebalancing orchestrator recalculates Rebalancing vBucket 4 vBucket 10 the vBucket map (including replicas) vBucket 5 vBucket 11 • Migrate vBuckets to the new server vBucket 6 vBucket 12 • Finalize migration vBucket migrator vBucket migrator Client After vBucket 5 • Node 3 is balanced vBucket 1 vBucket 7 vBucket 2 vBucket 6 • Clients are reconfigured to talk to vBucket 8 vBucket 3 vBucket 9 vBucket 11 Node 3 vBucket 4 vBucket 10 vBucket 12 17
- 18. Data buckets are secure Couchbase “slices” Application user Web application server Bucket 1 Bucket 2 Aggregate Cluster Memory and Disk Capacity Couchbase data servers In the data center On the administrator console 18
- 19. Couchbase and Hadoop Integration • Support large-scale analytics on application data by streaming data from Couchbase to Hadoop – Real-time integration using Flume – Batch integration using Sqoop • Examples – Various game statistics (e.g., monthly / daily / hourly rankings) – Analyze game patterns from users to enhance various game metrics Flume memcached TAP protocol listener/sender Sqoop engine interface Couchbase Storage Engine 19
- 20. Agenda • Introduction • What is Couchbase Server? – Simple, Fast, Elastic – Technology Overview (Architecture, data flow, rebalancing) • Tribal Crossing Inc: Animal Party – Challenges before Couchbase • Original Architecture – Why Couchbase? • Simplicity • Performance • Flexibility – Deploying Couchbase • New Architecture • EC2 • Data Model • Accessing data in Couchbase • Product Roadmap • Q&A 20
- 21. Tribal Crossing: Challenges Common steps on scaling up database: ● Tune queries (indexing, explain query) ● Denormalization ● Cache data (APC / Memcache) ● Tune MySQL configuration ● Replication (read slaves) Where do we go from here to prepare for the scale of a successful social game? 21
- 22. Tribal Crossing: Challenges ● Write-heavy requests – Caching does not help – MySQL / InnoDB limitation (Percona) ● Need to scale drastically over night – My Polls – 100 to 1m users over a weekend ● Small team, no dedicated sysadmin – Focus on what we do best – making games ● Keeping cost down 22
- 23. Tribal Crossing: “Old” Architecture and Options ● MySQL with master-to-master replication and sharding – Complex to setup, high administration cost – Requires application level changes ● Cassandra – High write, but low read throughput – Live cluster reconfiguration and rebalance is quite complicated – Eventual consistency gives too much burden to application developers ● MongoDB – High read/write, but unpredictable latency – Live cluster rebalance for existing nodes only – Eventual consistency with slave nodes 23
- 24. Tribal Crossing: Why Couchbase Server? ● SPEED, SPEED, SPEED ● Immediate consistency ● Interface is dead simple to use – We are already using Memcache ● Low sysadmin overhead ● Schema-less data store ● Used and Proven by big guys like Zynga ● … and lastly, because Tribal CAN – Bigger firms with legacy code base = hard to adapt – Small team = ability to get on the cutting edge 24
- 25. Tribal Crossing: New Challenges With Couchbase ● But, there are some different challenges in using Couchbase (currently 1.7) to handle the game data: – No easy way to query data – No transaction / rollback ➔ Couchbase Server 2.0 resolves them by using CouchDB as the underlying database engine ● Can this work for an online game? – Break out of the old ORM / relational paradigm! – We are not handling bank transactions 25
- 26. Tribal Crossing: Deploying Couchbase in EC2 Web Server Apache ● Basic production environment setup Client-side Moxi ● Dev/Stage environment – Cluster Mgmt. Requests feel free to install Couchbase on your web DNS Entry server Couchbase … Couchbase Couchbase Cluster 26
- 27. Tribal Crossing: Deploying Couchbase in EC2 Web Server ● Amazon Linux AMI, Apache 64-bit, EBS backed instance Client-side Moxi ● Setup swap space Cluster Mgmt. Requests ● Install Couchbase’s Membase Server 1.7 DNS Entry ● Access web console http://<hostname>:8091 ● Start the new cluster with a … single node Couchbase Couchbase ● Add the other nodes to the Couchbase Cluster cluster and rebalance 27
- 28. Tribal Crossing: Deploying Couchbase in EC2 Web Server Apache Moxi figures out which node in the cluster holds data for a given key. Client-side Moxi ● On each web server, install Moxi Cluster Mgmt. Requests proxy ● Start Moxi by pointing it to the DNS Entry DNS entry you created ● Web apps connect to Moxi that is running locally memcache->addServer(‘localhost’, 11211); Couchbase … Couchbase Couchbase Cluster 28
- 29. Tribal Crossing: Representing Game Data in Couchbase Use case - simple farming game: ● A player can have a variety of plants on their farm. ● A player can add or remove plants from their farm. ● A Player can see what plants are on another player's farm. 29
- 30. Tribal Crossing: Representing Game Data in Couchbase Representing Objects ● Simply treat an object as an associative array ● Determine the key for an object using the class name (or type) of the object and an unique ID Representing Object Lists ● Denormalization ● Save a comma separated list or an array of object IDs 30
- 31. Tribal Crossing: Representing Game Data in Couchbase Player Object Key: 'Player1' Plant Object Array Key: 'Plant201' ( [Id] => 1 Array [Name] => Shawn ( ) [Id] => 201 [Player_Id] => 1 [Name] => Starflower PlayerPlant List ) Key: 'Player1_PlantList' Array ( [0] => 201 [1] => 202 [2] => 204 ) 31
- 32. Tribal Crossing: Schema-less Game Data ● No need to “ALTER TABLE” ● Add new “fields” all objects at any time – Specify default value for missing fields – Increased development speed ● Using JSON for data objects though, owing to the ability to query on arbitrary fields in Couchbase 2.0 32
- 33. Tribal Crossing: Accessing Game Data in Couchbase Get all plants belong to a given player Request: GET /player/1/farm $plant_ids = couchbase->get('Player1_PlantList'); $response = array(); foreach ($plant_ids as $plant_id) { $plant = couchbase->get('Plant' . $plant_id); $response[] = $plant; } echo json_encode($response); 33
- 34. Tribal Crossing: Modifying Game Data in Couchbase Give a player a new plant // Create the new plant $new_plant = array ( 'id' => 100, 'name' => 'Mushroom' ); $couchbase->set('Plant100', $new_plant); // Update the player plant list $plant_ids = $couchbase->get('Player1_PlantList'); $plant_ids[] = $new_plant['id']; $couchbase->set('Player1_PlantList', $plant_ids); 34
- 35. Tribal Crossing: Concurrency Concurrency issue can occur when multiple requests are working with the same piece of data. Solution: ● CAS (check-and-set) – Client can know if someone else has modified the data while you are trying to update – Implement optimistic concurrency control ● Locking (try/wait cycle) – GETL (get with lock + timeout) operations – Pessimistic concurrency control 35
- 36. Tribal Crossing: Data Relationship ● Record object relationships both ways – Example: Plots and Plants ● Plot object stores id of the plant that it hosts ● Plant object stores id of the plot that it grows on – Resolution in case of mismatch ● Don't sweat the extra calls to load data in a one-to- many relationship – Use multiGet – We can still cache aggregated results in a Memcache bucket if needed 36
- 37. Tribal Crossing: Migrating to Couchbase Servers First migrated large or slow performing tables and frequently updated fields from MySQL to Couchbase Web Server Apache + PHP Client-side Moxi MySQL memcached TAP protocol listener/sender engine interface Reporting TAP Client Applications Couchbase Storage Engine 37
- 38. Tribal Crossing: Deployment 38
- 39. Tribal Crossing: Deployment 39
- 40. Tribal Crossing: Conclusion • Significantly reduced the cost incurred by scaling up database servers and managing them. • Achieved significant improvements in various performance metrics (e.g., read, write, latency, etc.) • Allowed them to focus more on game development and optimizing key metrics • Plan to use real-time MapReduce, querying, and indexing abilities provided by the upcoming Elastic Couchbase 2.0 40
- 41. Agenda • Introduction • What is Couchbase Server? – Simple, Fast, Elastic – Technology Overview (Architecture, data flow, rebalancing) • Tribal Crossing Inc: Animal Party – Challenges before Couchbase • Original Architecture – Why Couchbase? • Simplicity • Performance • Flexibility – Deploying Couchbase • New Architecture • EC2 • Data Model • Accessing data in Couchbase • Product Roadmap • Q&A 41
- 42. Product Roadmap: Couchbase Server 2.0 • Mobile to cloud data synchronization • Cross data center replication US West Coast Data Center US East Coast Data Center Couchbase Couchbase Server CouchSync Server CouchSync CouchSync Couchbase Single Server Couchbase Single Server CouchSync CouchSync … … … … … … 42
- 43. Product Roadmap: Couchbase Server 2.0 • Replace Sqlite-based storage engine with CouchDB • Support indexing and querying on values • Integrate real-time MapReduce into Couchbase server • SDK for Couchbase server Membase Server 1.7 CouchDB 1.1 Couchbase Server 2.0 The world’s leading caching The most reliable and full- The fastest, most complete and and clustering technology featured document database most reliable database on the planet 43
- 44. Couchbase Product Download • Community Edition – Open source build – Free forum support • Enterprise Edition – Free for non-production use – Certified, QA tested version of open source – Case tracking and guaranteed SLA for production environments • Community Sponsors in China – Interested in being Couchbase Ambassador for Beijing? 44
- 45. Q&A Steven Mih, Couchbase Inc. (steven@couchbase.com) Bin Cui, Couchbase Inc. (bin@couchbase.com) 45
- 46. QClub QClub是由InfoQ中文站定期组织的一个线下技术交流活动，目的是让中高端技术人员 有一 个相对自由的交流思想和交友的平台。每次QClub只关注一个主题，邀请国内外 在该主题领域有研究的技术专家分享其经验，但QClub更注重讨论，因为它认为讨论 才是真理，从讨论中才能激发参与者的智慧和火花。 关注我们：weibo.com/infoqchina QClub专题页：http://www.infoq.com/cn/qclub/ 敏捷 运维 QClub SOA 桥梁 InfoQ 云计算 技术演讲 争鸣 Open Space Ruby 畅想 技术社区活动 沟通 日历 分享 .NET Java 社区 趋势 交流 话题 QClub 架构