Real World Use Case with Cassandra (Eddie Satterly, DataNexus) | C* Summit 2016
Download as PPTX, PDF3 likes1,699 views
IAG, a leading insurance group in Asia and ANZ, aims to create a unified view of customers despite using over 130 internal and external data sources. They face challenges in integrating this data across various brands and products, having struggled for more than two years with traditional tools. To address these issues, they are exploring telematics data collection and are considering Cassandra as a solution for better data management and processing.
Real World Use Case with Cassandra (Eddie Satterly, DataNexus) | C* Summit 2016
- 1. Eddie Satterly CTO & Co-Founder DataNexus @eddie_satterly Real World Cassandra
- 2. 1 Who is IAG 2 Single Customer View 3 Telematics 4 Why Cassandra 2© DataStax, All Rights Reserved.
- 3. © DataStax, All Rights Reserved. 3 Insurance Leader IAG is a group of top insurance brands across Asia and ANZ region Whether you are a customer, partner, employee, shareholder or part of the communities IAG serves across Australia, New Zealand or Asia, IAG exists to ‘make your world a safer place’. The Group’s Vision is: To be Asia Pacific's most loved, most inventive and most successful insurer.
- 4. Data Challenge #1 © DataStax, All Rights Reserved. 4 Problem Statement #1 – Customer View As a group with a large number of brands and all with separate products most with separate source systems it is very difficult to represent a single customer in a useful way. There are over 130 internal source systems with data and several external sources as well that make up what IAG knows about customers, but no way to bring that data together. The view to the right is a high level overview. This is a much harder problem to solve than you might think and there has been more than 2 years of effort put into trying various approaches using traditional data tools. All have failed
- 5. How to Solve for Customer View © DataStax, All Rights Reserved. 5 SQLfor21Attributes +SystemKeys Column Families SCV API PostgreSQL Landing Zone Source 1 Source 2 Source 3 Source 4 Source 5 Source 6 Source 7 SourceX Customer Keyspace MDM (Matching) SQL Use Keys to Fetch Full Details Write Full Details Write Golden Record Customer Masters PIICustom erM asters API Contract: Allows Keys each to pull PII or Anon or Full Dataset Allows update submission Allows Read from Pending Updates UpdateSubmissions Readfrom PendingUpdates FullCustom erDetails Data Curators Data Stewards Updates Matched PII Full Anon . . AnonymousCustomerMasters Search UI
- 6. Data Challenge #2 © DataStax, All Rights Reserved. 6 Problem Statement #2 – Telematics Collection of massive amount of telemetry from devices in cars and homes provides a very rich source of information to make critical Insurance decisions and get our customers a better price and a better experience when they opt in to share. The initial use case was simply to pull in data from a few vehicles and determine if customers were interested in sharing that data to improve their insurance price in ANZ like they are in the US. Since then there has also been a project undertaken to help save on fuel and make drivers safer, in staying with IAG vision statement. On top of all this there is also a current effort to work with customers on connected home platform to price more accurately on home policies as well.
- 7. How to Solve for Telematics © DataStax, All Rights Reserved. 7
- 8. Why Cassandra © DataStax, All Rights Reserved. 8 Cassandra Solution Approach All on Openstack Appliance 9 Nodes Cassandra with 3x800GB SSD 3 Nodes Solr with 3x800 SSD All R3 2 node Kafka Broker for all data feeds 6 nodes Storm for processing and enrichment
- 10. Table Design Style Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content Content © DataStax, All Rights Reserved. 10