Gyula Fóra - RBEA- Scalable Real-Time Analytics at King
2 likes655 views
The document discusses the RBEA platform developed by King, which utilizes Apache Flink for scalable real-time analytics, processing 30 billion events per day from 463 million users. It outlines the architecture, stateful processing, and the use of a domain-specific language (DSL) for event handling. The platform is designed to simplify streaming analytics for data scientists, making it more accessible and powerful.
![© King.com Ltd 2016 – Commercially confidential
User states
Page 14
RBea
Script
LAST_GAMESTART
LAST_PURCHASE
LAST_…
+ User defined states
Fields
byte[]
User State
byte[]byte[]
LRU Cache
Scripts can read all state
But write only their own](https://image.slidesharecdn.com/gyulofora-rbea-scalablereal-timeanalyticsatking-160919123732/85/Gyula-Fora-RBEA-Scalable-Real-Time-Analytics-at-King-14-320.jpg)
Gyula Fóra - RBEA- Scalable Real-Time Analytics at King
- 2. © King.com Ltd 2016 – Commercially confidential RBea: Scalable Real-Time Analytics at King Gyula Fóra Data Warehouse Engineer Apache Flink PMC Page 2
- 3. © King.com Ltd 2016 – Commercially confidential We make awesome mobile games 463 million monthly active users 30 billion events per day And a lot of data… Page 3 About King
- 4. © King.com Ltd 2016 – Commercially confidential From streaming perspective… Page 4 DB 30 billion events / day Complex processing logic Terabytes of state DB DB
- 5. © King.com Ltd 2016 – Commercially confidential How do we use Flink Page 5 Standalone deployment Few heavy streaming jobs RocksDB state backend with caching Lot of custom tooling
- 6. © King.com Ltd 2016 – Commercially confidential The RBea platform Page 6 Scripting on the live streams Deploy from browser/python Window aggregates Stateful computations Scalable + fault tolerant
- 7. © King.com Ltd 2016 – Commercially confidential RBea architecture Page 7 Events Output REST API RBEA web frontend Libraries http://hpc-asia.com/wp-content/uploads/2015/09/World-Class-Consultancy-Seeking-Data-Scientist-CA-Hobson-Associates-Matthew-Abel-Recruiter.jpg Data Scientists
- 8. © King.com Ltd 2016 – Commercially confidential RBea backend implementation Page 8 One stateful Flink job / game Stream events and scripts Events are partitioned by user id Scripts are broadcasted Output/Aggregation happens downstream S1 S2 S3 S4 S5 Add/Remove scripts Event stream Loop over deployed scripts and process CoFlatMap Output based on API calls
- 9. © King.com Ltd 2016 – Commercially confidential Dissecting the DSL Page 9 @ProcessEvent(semanticClass=SCPurchase.class) def process(SCPurchase purchase, Output out, Aggregators agg) { long amount = purchase.getAmount() String curr = purchase.getCurrency() out.writeToKafka("purchases", curr + "t" + amount) Counter numPurchases = agg.getCounter("PurchaseCount", MINUTES_10) numPurchases.increment() }
- 10. © King.com Ltd 2016 – Commercially confidential Dissecting the DSL Page 10 @ProcessEvent(semanticClass=SCPurchase.class) def process(SCPurchase purchase, Output out, Aggregators agg) { long amount = purchase.getAmount() String curr = purchase.getCurrency() out.writeToKafka("purchases", curr + "t" + amount) Counter numPurchases = agg.getCounter("PurchaseCount", MINUTES_10) numPurchases.increment() } Processing methods by annotation Event filter conditions Flexible argument list Code-generate Java classes => void processEvent(Event e, Context ctx);
- 11. © King.com Ltd 2016 – Commercially confidential Dissecting the DSL Page 11 @ProcessEvent(semanticClass=SCPurchase.class) def process(SCPurchase purchase, Output out, Aggregators agg) { long amount = purchase.getAmount() String curr = purchase.getCurrency() out.writeToKafka("purchases", curr + "t" + amount) Counter numPurchases = agg.getCounter("PurchaseCount", MINUTES_10) numPurchases.increment() } Output calls create Output events Output(KAFKA, “purchases”, “…” ) These events are filtered downstream and sent to a Kafka sink
- 12. © King.com Ltd 2016 – Commercially confidential Dissecting the DSL Page 12 @ProcessEvent(semanticClass=SCPurchase.class) def process(SCPurchase purchase, Output out, Aggregators agg) { long amount = purchase.getAmount() String curr = purchase.getCurrency() out.writeToKafka("purchases", curr + "t" + amount) Counter numPurchases = agg.getCounter("PurchaseCount", MINUTES_10) numPurchases.increment() } Aggregator calls create Aggregate events Aggr (MYSQL, 60000, “PurchaseCount”, 1) Flink window operators do the aggregation
- 13. © King.com Ltd 2016 – Commercially confidential Aggregators Page 13 long size = aggregate.getWindowSize(); long start = timestamp - (timestamp % size); long end = start + size; TimeWindow tw = new TimeWindow(start, end); Event time windows Window size / aggregator Script1 Script2 Window 1Window 2 NumGames Revenue W1: 8999 W2: 9001 W1: 200 W2: 300 MyAggregator W1: 10 W2: 5 Dynamic window assignment
- 14. © King.com Ltd 2016 – Commercially confidential User states Page 14 RBea Script LAST_GAMESTART LAST_PURCHASE LAST_… + User defined states Fields byte[] User State byte[]byte[] LRU Cache Scripts can read all state But write only their own
- 15. © King.com Ltd 2016 – Commercially confidential Things you might wonder… Page 15 Can slow scripts affect other scripts? Yes, but we are working on it Separate test/live environments What does the backend know about the scripts? Outputs produced Failures (causes) => these are propagated Runtime stats in the future Is RBea useful compared to custom Flink jobs? Writing + maintaining streaming jobs is hard Especially with state, windowing, MySQL etc.
- 16. © King.com Ltd 2016 – Commercially confidential Page 16 RBea physical plan
- 17. © King.com Ltd 2016 – Commercially confidential Wrap up Page 17 RBea makes streaming accessible to every data scientist at King We leverage Flink’s stateful and windowed processing capabilities People love it because it’s simple and powerful
- 18. Thank you!