Tera stream for datastreams

Data Integration as Data Infrastructure

content Why TeraStream™?

Case Studies

Company Profile

Q&A


Data Integration
As
Data Infrastructure


Enterprise Data Management Suite
Minimal implementation effort and producing high-quality
Business Intelligence output.

Real Time Monitoring
• Performance Tuning

• Emergency Management

Source Reports
Systems Integrated ODS / DW Report
Mart
MERGE SPLIT Aggregation Calculation

ETL / Real Time / Near Real Time

SORT JOIN Conversion Reformat

Data Governance Architecture
Master Data
Meta Data Data Quality Impact Analysis
Management


Possession of Key Technology


DataStreams Products

ETL Test Data Security Change Data Capture Near Real Time

Meta Data Data Quality Master Data Management Impact Analysis

Common Benefit of DataStreams Products
 Simple Mgt. & Upgrade  Stability
 TCO Reduction  One-stop support
 Saving time cost by Speed  Extreme Performance

Why TeraStream™ ?


Floods of Massive Data
To get Business intelligence on the Data.. Need Data
Integration

ERP Systems Other Operational Apps

Legacy Systems Web Logs etc

Call Center Apps RDBMSs

File based information Unstructured Data


Data Integration Landscape
Business Challenges

Impact
Factors Result
 Multiple versions of the
 Disparate data  Bad decisions
“Truth”
 Inaccurate data  Wasted time and  Lost revenue

 Incomplete data resources aggregating
 Lost productivity
information
 Untimely data  Lost market opportunity
 Difficult to use Data
 Fragmented  Delayed Decision making  Bad Citizen relationships
Integration Approach  Uninformed management

This is more than 30 percent of corporate
IT budgets so data integrity is used to
emphasize what is important.


Well Positioned
TeraStream is very well positioned in the Data Integration
marketplace.
Powerful Performance (TeraSort™)
High-speed extraction (FACT™)
Reuse of data (ETL and Batch Hub)

Over 200 Customers
Serving multiple industries including banking,
government retail
Over a decade of experience

Intuitive GUI
Easy to Operate
Easy to maintain

Fast implementation
Easy customization
Low resource use


TeraStream™ Positioning
 A TeraStream™ bundle, one Unload tool, one Sort tool and ETL metadata
repository
 An end to end, High Performance driven data integration platform.
• Basel II
• Risk Management
• Data Warehouse
• Data Migration
• Data Synchronization
• Batch Execution
• Master Data Management


TeraStream™ Data Integration
Extraction, Transformation, Load with High performance
SORT Engine

Data extraction from various High Performance SORT engine resolves time bottleneck due to transform Automatic generation of scripts
can be used for loading to various
commercial DBMS in high speed
LARGE DATUM DBMSs

Text Text
Data Data
MERGE SPLIT Aggregation Calculation

High Performances SORT Engine

SORT JOIN Conversion Reformat

EXTRACT TRANSFORM LOAD


TeraStream™ Data Integration Benefits
3times Elapsed time improvement against its competitor
Result on Data Migration in Shinhan Bank, Korea

TeraStream™ IBM DataStage

FILE → FILE FILE → DB
FILE → DB DB → DB

 thread MAX for sort =3  Parallel = 4
 File manipulation : 35% CPU usage  File manipulation : 58% of CPU usage.
 Load : 80% of peak CPU usage  Load: 58% of peak CPU usage
Conclusion Conclusion
 Elapse time : 20 minutes  Elapse time : 59 minutes
 Wasted System Resource : 800  Wasted System Resource : 3000
( 40% Avg. CPU usage X 20 mins ) (50% Avg. CPU usage X 60 mins)


TeraStream™ Near Real Time Data
Integration Processing Benefits
High Performance improvement on Real Data processing
Result on DW Benchmark Shinhan Bank, Korea

(Minutes)
50
DataStage
40

30

10

03 TeraStream

02

01

0
1 10 100 200 1,000 2,000
(Ten thousand records)


TeraStream™ Large Data Benefits
High Performance improvement on Very Large Data processing
Result on Batch Job Benchmark National Health Insurance Corporations, Korea (2005)

55% Reduction 50% Reduction 90% Reduction

70day

11day 4day 7day
5day 2day


TeraStream™ Batch Processing Benefits
High Performance improvement on Batch Jobs execution
than DB processing
Result on Batch Job Benchmark Post Insurance, Korea

40,000 400,000 4,000,000
Records Records Records

12x

3x


TeraStream™ EBH Benefits
Massive volume of files extracted from source systems
are stored in EBH, and reuse in next step
ETL Time is reduced by avg. 56% In case of LG Telecom

Legacy ODS EDW MART

ORACLE 8.0.6 ORACLE Teradata Sybase

EBH (ETL Batch Hub)

EBH (ETL and Batch Hub)
stores temporary or result files which is shared for further table generation in EDW and DATA MART.


TeraStream™ Features
Intuitive User Interface

Intuitive task flow Metadata Repository
GUI for developers Impact analysis
Change history
manager

Scheduler Editor window
Scheduling by time / Mapping Creation
period / business
calendar

Configuration Management Project Version Execution log Web Monitoring File View


High speed data extraction with FACT unload engine

DBMS Supported SQL is supported High Speed Extraction

SQL

Database API optimized

Split data Filtering Export Format
 Time
 Timestamp
 Fixed / Variable
 CLOB / BLOB
 NULL
 FRAME
 FILL

Database Connectivity and Diverse manipulation process
and functions
DBMS Supported N:N mapping File to DB or DB to File

Built-in Function User Function Supported Function Types
 INSERT
 DELETE
 UPDATE
 INSERT - UPDATE
 DELETE – INSERT
 EBCDIC ↔ ASCII
 CLOB / BLOB
 NDB (Unisys 9-bit)
 HDB (IBM)


Uses various transportation method according to file
structure, security, amount of record and etc.
DBMS
File to DB data load for bulk data
 Provide Bulk Load to process large data
 Partition Load
Bulk Load
 Parallel Load

DBMS Load scripts automatically
 Automatically Generate Script by using DBMS
Load Script Schema information
Automatic  Fixed / Variable
 Provide DBMS Option

Encryption while transporting
 Data Readable Symmetric Encryption
 Maintain original data characteristic
 Support various encryption pattern


Case Studies


Kookmin Bank
EDW and integrated DM installation
 Various DBMS(IMS HDB, HOST DB2, Oracle, DB2 UDB) integration by using
Expected
TeraStream™
Result
 Meeting batch target time of 2 hours and 30 minutes for 4TB of EBCDIC data.
Issues System configuration
• M/F and IMS HDB conversion
File process flow DB QUERY
• Processing changed data in absence of time-series
column Source system EDW

• Processing large size data within batch process IMS HDB Sybase ASIQ 12.7
time(10TB/day based on source data)
• How to process high volume files in parallel ETL
A-SOR DM
Informover
Plans IBM M/F - Seg. split
- conversion & Array split
• Converting main frame data into data in Unix - logic applied

environment (10TB → 25TB) within 18 hours.
• Various data conversion and processing including HDB, DB2 ETL
Korean character conversion
- conversion
• ETL task from accounting system server to new ODW Informover
- logic applied
server(extracting appx. 200 GB of daily changed data
within 1 hour and 30 minutes by using FACT module of
Server RDB
TeraStream™)
TS(FACT) ETL
• ETL and Batch process in unified way.
• Batch job in core banking system within 6 hours. - Logic applied


Samsung Electronics
Global Database Integration using NRT ETL
 Efficiency is maintained despite cross transportation
Expected  Bi-directional NRT integration allows the same service regardless of system type
Result and country (Time from extraction to loading is 20 minutes.)
 Bi-directional remote data transportation using TeraStream

Issues System Configuration
• Registration in one country should have the same Smart Phone Smart Phone System in China
service at other country. System in
Germany
• duplicated record should be avoided due to cross
transportation.
DBs in DBs in
• 20 minutes Near Real-time Service Service
• Perfect Recovery scheme should be presented

Plans Sam To DB
UPSERT
• Rea-time data transportation between Germany and SAM To DB
NRT Extract NRT Extract UPSERT
China.
• Bi-directional synchronization between TeraStream of
Germany and China.
• 20 min. MAX loading time for transported data is
implemented using TeraStream NRT. 프로그램 성공, 실패 등 실행 결과

Web Monitoring
• Web Monitoring is developed


LG Telecom

LG Telecom new billing system data transfer
 The working hours shortened to D +3 and D +1 in reducing the system load
Expected
 On average, 56% of the effect of reducing working hours
Result
 Emergency response system rework due to delay in securing and providing data
to minimize Impact
Issues System Configuration
• Solution provided by „I‟ company requires more than
twelve hours for processing every billing and call data.
ETL EDW Server
• It delays entire processes and often requires re-
processing of data. D+1
• Efficient unique key generation for entire business tasks Custo
Call
Data Billing
Legacy ODS Server mer

Plans Contacts PPS/
BSS

• Transition from old to new billing system AR SRDF ODS
Mining
Input Campaign MOLAP
- Data size: 3TB→ 3.5TB, Object: Transition in Oracle Oracle/ Variables Analysis Analysis
Billing Informatica ETL
30 minutes Mining Teradata
Analysis
• Move unchanged data among large dataset three CSM CCS CTI NCR 10Node

days prior to the new system open date. MPS PPS
DM Server
• Separate files that will be loaded to EDW and DM ERP NMS
and load them in different business tables. OLAP
TeraStream loads data transformed Mart
• Unique key generation for entire business process
Sybase ASIQ
is done first. in ODS to EDW and DM at the same time. IBM P Series


Company Profile


Company Profile
Company DataStreams Corp.

CEO Young-sang Lee

Established 2001.9.19

Data Integration Solutions Development and Sales
- ETL/data migration/high performance batch product
-Deferred (Near Real Time) data integration solution
- Real time data integration product

Data Quality Solutions Development and Sales
Business -Metadata and data quality management solutions
- Application impact analysis product
 Master Data Management Solution
 Data Warehouse & BI Development / SNS Analytics
 Data Standardization & Quality Management Consulting
 High Performance Batch Job Technical Consulting

Investor JAFCO ASIA

Headquarter Chungho-Nais B/D 6F, 1597-3, Seocho-dong, Seocho-gu, Seoul, Korea
#601, U-Spacemall #2, Sampyoung-dong, Bundang-gu, Seongnam-si,
R&D Center
Gyounggi-do , Korea

Homepage www.datastreams.co.kr


Business Area
DataStreams provides “data integration and quality solutions” that enable your company to
stand firmly on concrete data infrastructure. It allows you to analyze and utilize your
internal or external data to gain competitiveness by empowering with timely, relevant and
trustworthy data for your top business imperatives. Based on leading data integration and
quality solutions, DataStreams has expanded its business area to Master Data Management
and Information Analytics as well as big data performance improvement, data migration,
various data integration and metadata management.

Data Integration Solutions Meta Data & Quality
Development / Sales Management Solutions
- Metadata and data
- ETL / Data Migration / High-speed Batch Tool quality management solution
- Real-time data Integration Product Business - Application impact analysis product
- Deferred data integration solution(NRT)
Areas Data Quality Consulting
D/W BI Development

Master Data High-speed batch
Management Solution & processing technical
Consulting SNS Analytics consulting


Mission of DataStreams
DataStreams is evolved toward data management company based on over 13 years of
experience in data integration and quality management. DataStreams helps information
users to build concrete data infrastructure for integration and quality management. Based
on superb quality of on-hand data, DataStreams‟s mission to satisfy customers by raising
enterprise value and competitiveness will be accomplished.

No.1 end-to-end data integration and quality management solution provider
Help your enterprise by providing data governance capability Consult and build
systems how to utilize your accumulated data for long years


Market Share & Reputation
DataStreams is No.1 data integration and metadata management solution provider in Korea. DataStreams
has been selected for 30 Korean software company worthy of global recognition in 2012.

Market share Good Reputation

No. 1 Share in Domestic Market

Etc 6%
SAS 3%
SAP 5% DataStreams
ORACLE 28%
6%

INFOMATICA,
10%

Wise
IBM 9%
10%
Encore Gtone
BTL
7% 10%
6%

<Domestic M/S>
* Domestic M/S is researched by DataStreams (2008)
• M/S of Global Vendors is quoted by "Worldwide DI and Access Software 2006
Vendor Shares: Market and Segments," IDC.


Credentials
DataStreams is recognized as leading company in data integration & quality management in domestic
market. DataStreams is the only solution provider who has expertise in both solution market. DataStreams
has developed its own methodologies for master data management and data governance technologies for
EDM. As a leader, DataStreams owns good references in various industrial fields (especially, more than 80%
of market share in domestic banking industry).

Data Integration Data Quality Methodology & Experiences

Data quality technologies with metadata
Data Integration technologies Its own methodology for MDM & lot
management, data quality and application
with ETL, CDC and NRT solutions. of technical experiences
impact analysis solutions.

Progress No. 1 Market Share Professional Support

DataStreams
80%
Perfect data governance for data
administrator is progressive approach for
EDM . It is based on product line-up from 80% of M/S in domestic Banking Market, Technical support and quality assurance
ETL to real-time and metadata to impact Market Share, ETL（55％）#１ in with Over 60 Consultants and over 34
analysis solutions, Data Migration（60％）#1 in Korea R&D personnel


DataStreams is the SW Market Leader in Korea
TeraStream™ has No.1 market share and stellar references in ETL and Migration market in Korea.

5%

15%

ETL &
55% Migration
25%

TeraStream IBM Data Stage Informatica Other

- 31 -

DataStreams dominates Korean banking industry
Most of major banks in Korea use TeraStream™ as their major ETL product. TeraStream is
unique market leader in Korean banking industry.


The Market Leader in Korean is going global
DataStreams already has a strong global presence with many of its products and plans to roll
out its entire data management product suite internationally

Merklenet.Inc.
Accenture Gmbh Airweb
CSC Consulting
Sungard
Bisys American Air

Telefonica Soluciones Reliance Industry Comcast Highmark, Inc

Procecard Excelcom Merkle Data Tech
Techocom
Sodexhopass
Telkomsel
Aviva

Banco Colpatria

Credibanco


Strong Track Record of Success
2012 05 Open Pangyo Techno valley R&D Center
2005 04 TeraStream™ won New Technology Certification
from Ministry of Knowledge Economy
2011 12 Datastreams China. established
04 Acquired KDB Solution Co., Korea‟s first meta-data
11 Received SW achievement award by Korean Government management solution company.
06 Selected as an ATC(Advanced Technology Center) 03 Awarded for Product Competitiveness from SMBA

2010 12 TeraStream™ V.3.2 released
2004
08 TeraStream™, trademark registered (No.0589927)
03 CEO, Young S. Lee elected as chairman of 05 First worldwide sales contract of FACT™
KOSEA(Korea Software Enterprise Association). 04 Selected as technically innovated company of
01 MetaStream™ V.2.7 released 2004 by Small & Medium Business Administration

2009 06 QualityStream™, ImpactStream™ released
2003 10 Presented FACT™ to Oracle Open World 2003 in
San Francisco, USA
02 DeltaStream™ V.1.1 released
08 KEB selected TeraStream™ as standard Batch/
06 Contracted with Intellectual Property Office ETL solution for Next Generation Banking System.
2008 for Enterprise Data Quality Management.
03 Company‟s name change to DataStreams 2002 12 First TeraStream™ V.1.1 contract at National
Statistics Office
11 JAFCO, Japan invested 4 million USD. 07 TeraStream™ exposed in SEK2002, Seoul
07 Awarded for Excellent Venture Company by 06 TeraStream™ Version 1.1, Released
2007 Deputy Prime Minister 01 Registered TeraStream™ as a trademark
06 TeraStream™ Version 2.0 released
05 MetaStream™, awarded for Digital Business 2001 09 Innovative Data Solutions Corp., established
Innovation by SMBA

2006 12 TeraStream is selected as standard ETL for 16 2000 04 TeraStream development started
01 CoSort was successfully applied to a DW
Province Public Administration Office
project for the first time at LG company
11 TeraStream™ V.1.4 won GS(Good Software)
Certification 1999 09 CoSort Korea, established


Marquee Accounts in Korea and Beyond
Banks

Insurance

Securities

Card/
Finance

Government/
Public Sector/
Universities

Telco

Manufacturing
/Services


Thank you
www.datastreams.co.kr

Tera stream for datastreams

More Related Content

Similar to Tera stream for datastreams (20)

Recently uploaded (20)

Tera stream for datastreams

Editor's Notes