Tapdata Product Intro
- 1. A Smart Data as a Service Platform
- 2. Beer On Tap Fresh beer, ready to serve Real time data, ready to serve Data On Tap
- 3. page 03 01 Our Opinion 03 Use Cases 02 Our Product Table of Contents 04 Customer References
- 4. page 04 01 Our Opinion Data is asset Challenges of data value realization What is Data as a Service
- 5. page 05 Our Opinion Data is asset DATA == VALUE Focus on and measure the value of your business data Consider data is measurable corporate asset Good directory management and definition Data doesn't have much value Realize data directly through API economy Gartner report 89% of CEOs believe that data is a measurable and computable corporate asset, and some even put on their balance sheet. Only 11% of CEOs consider that data doesn’t add much value.
- 6. page 06 Our Opinion Challenge: Data Silos, Governance, and Availability Don't care where the data is, how to store it, use it when you want to use it. Desirable Scenario Reality Check Data Data Data Data Data silos Cumbersome ETL Data consolidation Unable to quickly realize the value of enterprise data assets
- 7. page 07 Our Opinion Data as a Service: The Transition From Analytical to Operational Before Gain insights via big data analytics Offline Big Data Real-time streaming data Structured data Unstructured data Analysis Analysis Analysis Business decisions Risk control Customer Insights Data as a Service Streaming data Structured data Service Service Service Analytical Model Unstructured data Operational Model Transaction Operation Production Analysis Business innovation Third-party data External data Network data Communication data Credit data Customer data Sensor data Social media IT/OT Image video Today: Enable innovation with operational data service Business innovation Business innovation Business innovation Transaction Operation Production Analysis
- 9. page 09 Our Opinion DaaS: The Evolution of Data Architecture Business database 1990s Data lake10sData warehouse 00s Existing Operational Systems BI report Big data analysis Current enterprise data architecture Operational Data as a Service Platform 2020s Mobile/Web Applications Dashboard / Analytics Enabling business Enterprise Data Architecture 2.0
- 10. page 010 Our Opinion A e-Commerce DaaS Architecture (Alibaba ) Data asset management Asset analysis Asset catalog Asset governance Asset application Asset operation Unified Service Layer OneService Batch Ingestion | Real-time Sync | Streaming | File Collection Internal User and External User, Mapping, Auth OneID Data Development Data Quality Model Building Data Standard Data Synchronization Data Development Standard data model for operational consumption OneData Analytics Model Product Model Store Model Logistic Model Customer Model Monitoring
- 11. page 011 02 Tapdata Product Functional module features
- 12. page 012 - - - - - - Code-lessAPIServer Tapdata platform is a real-time big data service product, designed to help enterprise to accelerate the digital transformation journey. Tapdata product consists of 4 major components and capabilities: Data Synchronization and Collection Data transformation and modeling Scalable and Flexible Data Storage Code-less API Server - / Synchronization And Collection - -
- 13. page 013 StoreandProcessDataService Cluster Cluster Cluster Data Governance Security Charts ServerStreaming API SQL API Server-lessMongoDB API Collection Bulk Ingestion Service Real-time Heterogeneous Replication Marketing IOTData warehouse CustomerBusiness database Streaming data Data publish Data sharing Data Catalog Data modeling Data governance Rules verification File system API backend App backend Real-time Monitoring BI Report SQL compatible Real-time board Embedded visualization Server-less Application RESTful API Product Description Tapdata DaaS Architecture Overview
- 14. - - page 014 - - - - - - - - - - - - - - -- - - - -
- 15. page 015 Data collection 01 02 03 04 Product Description Data Collection Features Log based replication Agent-less Automatic data validation between source and target Single node 240GB/hour Multi-node deployment Oracle, MySQL MSSQL, Sybase Excel, XML, PDF/Word Real-time synchronization Minimum delay Data consistency guarantee Distributed setup High concurrency Multiple data source Heterogeneous database support
- 16. page 016 Product Description Data Collection Screenshot
- 17. page 017 Product Description Data Collection - Create Data Source
- 18. page 018 Product Description Data Collection - Job List
- 19. page 019 Product Description Data Collection - Job Edit
- 20. Product Description Data Governance and Modeling Module page 020 Data processing Merge & split Intelligent Data Governance Calculation enhancement Type conversion Data cleaning Data quality Rule check Dirty data detection Data rules Quality statistics AI modeling* Data Catalog Tag Management AI data catalog* * Planned feature in future version
- 21. page 021 Product Description Data Catalog - Open Data
- 22. page 022 Product Description Data Catalog - Data Quality Rules
- 23. page 023 Mapping Designer The system automatically recommends mapping based on the relational structure. Users can also customize the mapping rules from relational to JSON structure in an intuitive way, and provide instant JSON structure preview. In the future, based on cloud modeling data, AI technology will be used to automatically recommend practical models. Target JSON Relational tableProduct Description Relational to Big Data Modeling
- 24. page 024 Data Storage Application Driver mongos Primary Secondary Secondary Shard 1 Primary Secondary Secondary Shard 2 … Primary Secondary Secondary Shard N … …mongos Product Description Scalable Data Storage HTAP Support ● Support different workloads in one cluster ● OLTP & OLAP Multi-mode Database ● Structured, Relational, XML, JSON ● Semi-structured data, Log, Text, etc ● Unstructured, PDF, word, image Auto-Scale ● TB – PB data volume ● No downtime scale ● Many thousands concurrent users ● Workload Isolation ● Geographical Deployment High availability ● Automatic replication of data between cluster members ● 99.999% high availability ● Active-Active Multi-DC deployment Application Driver Application Driver ConfigServer Config Server Config Server
- 25. page 025 Product Description Data Publish Data distribution Shard1 Shard2 Shard3 Shard4 API DesignerAPI Security API Monitoring Process management* API Stats API Documentation API Server API Server Tap API DaaS API Client Mobile application Data consumption Data Distribution
- 26. page 026 01 02 03 04 Product Description Data Publish Module Features Code less API design and publish Auto generate API documentation and test portal Capture all invocation log and provide detailed analysis Deploy on VM or Container Instant API Backend Documentation And Test API Log and Stats Automatic deployment Scale as needed
- 27. page 027 Product Description Data Publish - Create API
- 28. page 028 Product Description Data Publish - API List
- 29. page 029 Product Description Data Publish - Data Explore
- 30. page 030 Product Description Data Publish - API Test
- 31. page 031 Product Description Data Publish - API Stats
- 32. page 032 03
- 33. page 033 Government Open Data 03 • Government is the organization that produces the most data. Many departments have different shape of data, data exchange and distribution has always been a challenge. • With the capabilities of data collection, mass data storage, automatic cataloging, fast data publishing and security, Tapdata provides a fast data exchange platform solution. Relational to NoSQL 01 • Relational databases are facing scalability and performance challenges, Tapdata can help to migrate data from RDBMS to MongoDB in real time • Many enterprises adopt dual- mode IT strategy. Keep the existing IT solutions unchanged, but replicating data to a new platform to embrace new technology and enable innovation. Tapdata helps to connect the two ID mode. Data as a Service Platform 02 • Connect the data silos, consolidate enterprise data using unified and standard data model, bridges the gap between data and value, creating a service- oriented data platform. • Tapdata uses real-time data collection tools to aggregate data into platform, and provides RESTful API to application developers, which greatly increases the speed of time to market for new applications Use Cases Application Scenarios
- 34. page 034 API Economy 04 • Data and processes are delivered in an API manner under the micro-service architecture, and more enterprises directly realize the value of data via API • The API designer, API automatic publishing and traffic monitoring functions of Tapdata provide out- of-the-box solutions for enterprise data publishing, which helps enterprises realize data value quickly. IOT Data Center 05 • IOT data has the characteristics of high frequency, large throughput, variable data types and high real-time requirements. . • Tapdata can provide PB-level data storage, second-level data access and analytical capabilities to meet real-time IoT use case requirements. Enterprise Content Management 06 • The traditional ECM system based on Filenet has faced the challenges of insufficient capacity, slow performance and difficult backup management. • Tapdata provides flexible storage structure based on MongoDB distributed elasticity and expansibility, as well as concise and easy-to-use data import and complete cataloging management capabilities, which perfectly solves the requirements of storing and managing massive number of files. Use Cases Application Scenarios
- 35. page 035 04
- 36. Real Time, Hybrid Replication RDBMS to MongoDB
- 37. Scenario Current Situation Tapdata Solution Difficult to scale, High cost Difficult to change Core System Use CQRS, Data Replicate to distributed database, Develop on it 1. Migrate Relational Data from RDBMS to MongoDB 2. Batch import for one time/initial sync 3. Real time replication via Tapdata CDC 4. Transform relational schema to MongoDB JSON 5. A ready to use operational data model in MongoDB Relational Database of Single instance, difficult to support Big Data IoT, new CRM need flexible Data Model Customer Cases Relational Database - MongoDB RDBMS Migration Database Modernization
- 38. page 038 PDF Web Mobile EPF CMS (…) APIs Tapdata Replicator ... APPT CMS EPR (…) Sybase Sybase Sybase Sybase Slave CDC < 1s delay MongoDB MongoDB MongoDB MongoDB MongoDB MongoDB Tapdata API Service
- 39. Finance DaaS A Local Retail Bank
- 40. B - / - - - 2 - 0 Real-Time Data Enquiry Real-Time Data Source Integration Legacy Systems Offloading Real-Time Reporting Data APIs / Open APIs Data UX - - / / v - - 0 - D 0 .- A / v - A / v - - 0 B B 2 0- - . A - - 0 .- / 0- - v A - - /- 0 A . 2-/ - . -// 2 0- - 2 / v . - - / - 0 - - B -0 -
- 41. page 041 Customer Cases Customer Data Model in DaaS Platform Basic demographic, socio-demographic information around the customer, including internal relationship info with the bank and external relationship info with his/she social groups Personal data Personal information Financial information Customer Behavioral information Transaction data Relation- ships Customer profile • Basic personal info • Health information • Customer identification • Channel preference • Stickiness & frequency • Channel behavior • Campaign behavior • Customer category • Price sensitivity • Customer satisfaction • Risk appetite • Key events • Transaction • Action triggers• Positions • Margins / profitability • Commissions • Asset products • Liability products (on/off balance sheet) • Insurance funds • Cards • AML • Basel • Taxation Products Credit risks Compliance • Social relationship • Interest group • Socio-demographic info Key Data Domains and Sample Data Categories Customer behavioral information, including transaction and interaction with the bank along all the touch points, covering RFM (recency, frequency, and monetary value) Behavioral data Customer financial assets with the bank and potentially outside the bank in the border ecosystem, including credit & risk rating, and financial product portfolio Financial data • Credit ratings • Risk score • Pricing associated • Guarantees • Recovery and collections
- 42. page 042 Customer Cases Real-time Enquiry and Core System Offloading Source Data Data Streaming and Transform Data Serving Layer Application Layer Consumer Layer Real-time Transaction Batch Balance, Transaction BDD ISB SFTP DaaS Application Interface Mobile Banking Internet Banking Call Center Branch
- 43. & & & & &
- 44. page 044 , C CFA , , • C CB M CB IF CB CB • IDDCF .F ,CB C 10+1 FJ F • IDDCF L 1 1.- ,+ • FC C C IDDCF • CF B ,CB C • : N • 1 J F : MD C B CF • CJ FB B • I CB B B A B • F B CFA CB • F F B A • AC: B • : 1.- AC: • I CA : A DD B FI • I AC: B • EI M B: C • A B A B • B M B: : ACB F CB C : EI M • DI • I : B : C DI • -C C: • I C DI
- 45. page 045 B 3 3 31 3 13 3 3 3 B 32 3 3 1 3 C 3 B 3 3 3 1 3 C 3 313 1 3 C 3 B B 3 1 3 3 2 B 3 3 3 B 3 3 3
- 46. Government Data Exchange Platform Shanghai Credit Data Center
- 47. page 047 Customer Cases Government Data: Exchange , Open Data, Decision Current: Data Island Phase 1 Data Exchange Platform(internal) Phase2 Data Open Platform(public) Phase 3 Government analysis and decision-making platform Data collect Real time replication Relational data Non-relational data One collect from all department Data governance Data catalog Storage classification Data catalog management enhancement Complete data catalog management Internal data API Data sharing Batch data download API management Security Full text index Innovative application data API Part of data Non-real time data All data Real time data API payment Data-driven decisions Data mining Deep learning AI decision
- 49. Educational Data Development Platform
- 50. page 050 Video Resource Educational System Data OA Teacher Development School Wechat Book Management Study Behavior Analysis Net Disk Others Description Live education Basic data from teacher, student, parent, school Regulation Announcement application, Resource utilization etc. Teachers’thesis Editing papers, Open courses, Lectures, Teaching competition and student mentoring program etc Wechat API for various educational system Book borrowing system. Purchasing management Students’ study behavior analysis system File management service Educational Management system, Financial management system, Equipment asset management system Applicable Level Regional Regional Regional Regional Regional Partial School Partial School Regional Provincial, Municipal Service Provider NaJia KeDa WeiWang WeiYan Tencent SiDanMei HaiKang Eisoo ......
- 51. page 051 - ( Development ) EEE EEE ( EEE ( B - / -Images A A C C D A - A single data source for future application - A complete data source for accurate data reporting - 5 times faster to build new application - 10 times faster to generate report
- 52. ( ( ( / ( )/ ( ( ( . () ( ( ( . ( ( ( ( .