Kai Wähner – Real World Use Cases for Realtime In-Memory Computing - NoSQL matters Barcelona 2014
1 like1,536 views
The document discusses in-memory computing, highlighting its role in real-time data processing and its various applications across industries. It emphasizes the advantages of in-memory solutions over traditional methods, especially in terms of speed and efficiency for business operations. Key use cases include personalized customer experiences, inventory management, and real-time event handling in sectors like telecommunications and logistics.
Kai Wähner – Real World Use Cases for Realtime In-Memory Computing - NoSQL matters Barcelona 2014
- 1. In-Memory Computing “Real World Use Cases” Kai Wähner Technical Lead kwaehner@tibco.com @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!
- 2. Kai Wähner Consulting Developing Coaching Speaking Writing Selling Main Tasks Requirements Engineering Enterprise Architecture Management Business Process Management Architecture and Development of Applications Service-oriented Architecture Integration of Legacy Applications Cloud Computing Big Data Contact Email: kontakt@kai-waehner.de Blog: www.kai-waehner.de/blog Twitter: @KaiWaehner Social Networks: LinkedIn, Xing
- 3. Disclaimer ! These opinions are my own and do not necessarily represent my employer
- 4. Key Messages In-Memory Computing is used for Acting in Real-Time! In-Memory Computing is NOT just Caching! Eventing and Fault-Tolerance move In-Memory to another Level!
- 5. © Copyright 2000-2014 TIBCO Software Inc. 5 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
- 6. © Copyright 2000-2014 TIBCO Software Inc. 6 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
- 7. Time Business Value Business Event Data Ready for Analysis Analysis Completed Decision Made $$$$ $$$ $$ $ Action Taken Business Value of Events over Time In-Memory Computing and Event Processing speeds action and increases business value by seizing opportunities while they matter
- 8. Drivers for In-Memory Computing • Hardware costs declining • Data Processing Requirements exploding • Traditional Approaches not scaling © Copyright 2000-2014 TIBCO Software Inc. 8 – Relational Databases – Clustered Databases – In-Memory Caches – Messaging Systems
- 9. © Copyright 2000-2014 TIBCO Software Inc. 9 Database Landscape in 2014 h9p://blogs.the451group.com/ informaCon_management/2014/03/18/ updated-‐data-‐plaIorms-‐landscape-‐ map-‐february-‐2014/
- 10. © Copyright 2000-2014 TIBCO Software Inc. 10 Agenda • Introduction to In-Memory Computing • Use Cases / Customer Success Stories
- 11. Caching for Fast Data Access LOADER • Cache to slower systems • Read-‐only • Not the system of record • No persistence required • Side benefit: Backend load is reduced
- 12. Caching + Dynamic Load SERVICE • Dynamically loaded into Memory when the data is first accessed by a client applicaCon • Service can present a standard interface • Client applicaCons are not required to implement any In-‐Memory specific code (1) Check Cache (2) Load from DB if not in Cache
- 13. Routing Messages to Back-Office Applications • Receive a common data feed that needs to be parsed and routed to several back-‐office applicaCons can use • In-‐Memory holding reference informaCon for the rouCng applicaCon. The router can quickly determine where to send the data. • Examples: Bank payments, insurance claims processing
- 14. Off-loading expensive systems Expensive in terms of response Cme and / or transacCon costs!
- 15. Personalized Customer Experience “With 38 million fans, MGM knows how to put its customers first, it takes more than a smile too. Customers want a personalized, tailored experience, one that knows their name and can anCcipate their needs. With the help of TIBCO technologies that leverage big data and give customers a digital idenCty, MGM can send personalized offers directly to customers, save them a seat, and have their favorite drink on the way. With mulCple customer touch points and channels, MGM can reach customers in more ways, and in more places, than ever before.” h9ps://www.youtube.com/watch?v=X-‐7S3kCOx9k Latency Problems: • Several Legacy Systems • Processing via ERP, CRM, Host, etc. In-‐Memory: • Enable Real Time • Only customers that have checked in • System of Record
- 16. Handling temporary spikes on a slow ‘system of record’ • An In-‐Memory event listener gets noCfied whenever a data value is changed and sends updates through a message queue for updaCng the master system of record. • The back office system can also be updated through other channels. • Examples: Christmas Shopping in E-‐Commerce, Ticket Sales, Online Bekng
- 17. Operational Data Store (Local File System) à In-‐Memory as “system of record” à OpConal: PersisCng data on the local file system (rather than requiring a database for persisCng data
- 18. Operational Data Store (Local File System) • Low-‐latency, high-‐throughput operaConal data – Customer data: e.g. account status and balance, purchase history: real-‐Cme loyalty (promoCons, cross-‐selling), fraud detecCon, ... – Market data: e.g. risk assessment, porIolio mgmt, producCon output opCmizaCon, buyer-‐seller matching – Sensor data: e.g. smart metering / grid, public transport safety – Track and trace: e.g. barcode scans, RFID: logisCcs, airlines • Why In-‐Memory? – Much faster than tradiConal DB, especially many small transacCons (XTP) – State / data management not addressed by messaging soluCons – EvenCng is a first class feature, changes can be ‘pushed’ in real-‐Cme to interested parCes (subscribe to changes, conCnuous queries) – Provides for distributed process synchronizaCon – Integrated with CEP engine (e.g. TIBCO BusinessEvents)
- 19. Situation Retailer: Inventory Management • Master data management system stores over 800 million customer records across more than 30 enterprise apps. • Stores real-time inventory data to enable ‘Buy online and pick-up at store’ and ‘Smart fulfillment’ features Problem • Due to lack of correlation between Point of Sale data and inventory, the website contained outdated inventory data. Products were listed as out of stock when there was actually inventory. • Need to leverage store inventory as well as inventory located fulfillment centers Solution • In-Memory stores real-time inventory data for the website, the fulfillment application, and other applications that need access to inventory data Business Impact • Reduction in customer churn • Intelligent fulfillments leading to greater customer satisfaction • Improved overall efficiency of fulfillment centers and store inventory
- 20. Distribution of Rapidly Changing Data à Examples are monitoring data for a power plant, stock market data, telemetry data for a complex system (example, a satellite), or the status and locaCon of packages for a major logisCcs or shipping company.
- 21. Telco: Real-Time Offer Generation and Fulfillment by Different Subcontractors Purchase 3G Package Cross-sell Voice/SMS package to subscriber who purchases 3G Mobile Package Total: 3 mio / day Peak: 50 events per sec Reload Give 100 free SMS to subscriber who tops-up Total: 12 mio top-up / day Peak: 300 top-up per sec Voice Call Give discount VOIP package to subscriber who makes a IDD call Total: 200 mio / day Peak: 12,000 events per sec SMS Usage Give discounted SMS package to subscriber who sends SMS more than 10 times a day Total: 750 mio / day Peak: 27,000 events per sec Purchase BB Package Event Cloud Reload Voice Call IDD Call OnNet Call SMS Usage Event Handling and Processing Touchpoint Integration Fulfill SMS Package Fulfill 3G Package Fulfill Voice Package Fulfill SMS Package Billing, Offer Fulfilled 46.7 million subscribers 2,000 SMS notifications per seconds 500 offer fulfillments per second Offer Message Reminder Message Fulfillment Message
- 22. Storing State-full Data for Enterprise Applications State-‐full Data
- 23. Super Fast Compute Grid for Intermediary Calculations for Analytics
- 24. Super Fast Compute Grid for Intermediary Calculations for Analytics • Technical issues in distributed grid compuCng with large scale data – Work load distribuCon – Process synchronizaCon – Data transfer • Examples – Risk assessment and management – OpCmizaCon problems: scheduling, cargo assignment, load distribuCon in power network / grid • Why In-‐Memory? – Many useful synchronizaCon features (e.g. atomic “take”) – LocaCon transparency and fault-‐tolerance – Real-‐Cme instead of nightly / weekly / ... Data-‐Warehousing approach
- 25. Key Messages In-Memory Computing is used for Acting in Real-Time! In-Memory Computing is NOT just Caching! Eventing and Fault-Tolerance move In-Memory to another Level!
- 26. Questions? Kai Wähner kwaehner@tibco.com @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!