Db2 analytics accelerator on ibm integrated analytics system technical overview

© 2017 IBM Corporation
Db2 Analytics Accelerator on IBM Integrated
Analytics System
Technical Overview
November 2017
Daniel Martin – danmartin@de.ibm.com

© 2017 IBM Corporation2
Disclaimer
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal
without notice at IBM’s sole discretion. Information regarding potential future products is intended
to outline our general product direction and it should not be relied on in making a purchasing
decision. The information mentioned regarding potential future products is not a commitment,
promise, or legal obligation to deliver any material, code or functionality. Information about
potential future products may not be incorporated into any contract. The development, release,
and timing of any future features or functionality described for our products remains at our sole
discretion.

Agenda
• Analytics Accelerator Version 7
• New architecture
• New engine
• API Compatibility to previous generations
• Deployment on IBM Integrated Analytics System
• Sister deployment on IBM Z
• Platform Comparison

Db2 Analytics Accelerator Version 7.1 Deployment Options
In Version 7.1, Db2 acceleration can be implemented within
different operating environments:
• On an on-premises appliance
• On a software appliance installed on the z14 mainframe
Both new options will offer
• the same functionality
• the same API
• the same implementation
This provides:
• Coexistence and combination of deployment options, fully
transparent for Db2 applications
• Flexibility in moving data for query acceleration as
workload demands grow or change
• Consistency and efficiency in managing different Db2
Analytics Accelerator environments
Next Generation Technology:
Two deployment options
Current Technology:
Appliance

Hardware Appliance
Uniform experience, simultaneous use, and easy transition between different implementations
Common analytics engine across all the platforms: Db2 Warehouse
Summary: One API – One implementation – Two deployment options
Deployment on IBM Z

Db2 Analytics Accelerator Version 7.1:
Shared Container

Version 7.1 - New Architecture
Virtual or physical server CPU Memory
Docker Supported OS
Storage (local, SAN, NAS)
(Clustered) Filesystem
Docker container
IBM Analytics
Engine
Authentication
Accelerator
server
Workload
Monitoring
Systems
Manager
Additional
future
functionality
Infrastructure
Management

Powered by Db2 with BLU Acceleration
•Huge potential for faster ingest for incremental updates, and thereby less HTAP query delay!
•IBM’s premier analytics engine across many products
•Latest analytics technology innovations
•Improved SQL compatibility and performance
•Higher degree of concurrent users and queries
•Roadmap: Spark, ML, Notebooks, DSM, …
In-memory column processing with
dynamic movement of data from storage
Multi-core and SIMD parallelism
(Single instruction Multiple Data)
Patented compression technique
preserves order so data can be used without
decompressing
Skips unnecessary processing of
irrelevant data
The new engine is replaced internally – external interfaces will stay the same.
The same Db2 subsystem can be connected to the existing and new generation.

SQL Coverage: Comparing Accelerator V7 with existing V5 Technology
Data Type support Accelerator V7 (on IAS or Z) Accelerator V5 (on PDA)
EBCDIC MBCS, GRAPHIC Supported natively Converted to UTF 8
TIMESTAMP(12) Supported natively Truncated to precision 6
FOR BIT DATA subtype Supported natively for EBCDIC, UNICODE,
ASCII
Supported for EBCDIC only
DECFLOAT On the Roadmap Not supported
BINARY On the Roadmap Not supported

SQL Coverage: Comparing Accelerator V7 with existing V5 Technology
SQL support
Accelerator V7 (on IAS or Z) Accelerator V5 (on PDA)
correlated subqueries All types supported including table expressions with
sideway references
Only a small subset supported
Recursive SQL Supported Not supported
TIMESTAMP value 24:00:00 Supported natively Mapped to 23:59:59.999999
Scalar functions Improved support Some not supported when using specific
datatypes, e.g. MIN/MAX, DAY, LAST_DAY,
BIT*, TIMESTAMP_ISO, VARIANCE/STDDEV/…
with UNIQUE clause,
HEX() function Supported Not supported
Mixed Encodings Adding EBCDIC tables when UNICODE tables
already added is supported
(SQL joins with data in Unicode and EBCDIC not
possible)
Only supported to add UNICODE tables after
EBCDIC table has already been added (required
to set AQT_ENABLE_MULTIPLE_ENCODINGS
environment variable)
CURRENT_TIME,
CURRENT_TIMESTAMP,
CURRENT_DATE
Supported with improved accuracy (no longer
dependent on time synchronization)
Supported
Local Date Exit format Not supported Supported

Db2 Analytics Accelerator Version 7.1,
Deployment on IBM Integrated Analytics
System

NPS®
8000 Series
TwinFin™ with i-Class™
Advanced Analytics
NPS®
10000 Series
TwinFin™
2003
2006
2009
2010
PureData System for
Analytics N2000
PureData System for
Analytics N3000
IBM Integrated Analytics System
(PDA & PDOA Convergence)
Convergence - Combining the best of Netezza and DB2 Appliances
2012
2014
2017
BCU
for AIX
BW
7000
BW
7050
BW
7100
BW
7100
ISAS
7600
ISAS
7700
ISAS
7710
PDOA
1.0
PDOA
1.1

Db2 Analytics Accelerator for z/OS Version 7.1, deployment on IBM
Integrated Analytics System (IIAS)
• Next generation hardware appliance
• A full solution that provides all components out of the box –including optimized hardware and software
• All components provided by IBM in a balanced, performance-optimized configuration
• HW, which includes the rack, the physical servers and the storage
• SW stack including the Docker host operating system as well as the Docker container and the
infrastructure management
• IBM Power hardware for the appliance, balanced and optimized for price/performance

Hardware Building blocks (Server/Storage)
Compute Node
 Power8 S822L
 Murano Chipset
 Two 12-core Sockets @ 3.42 GHz
 512 Gigabytes of DDR3 memory
 Four dual port 16Gb FibreChannel HBAs
 Two Ethernet NICs, each with two 1Gb/s
two 10Gb/s ports
 Two 600GB SAS hard drives
 Flash Storage
 IBM FlashSystem 900
 Dual Flash Controllers
 5.7 Terabyte MicroLatency Flash Storage
Modules
 Two-Dimensional RAID 5 and spare for
extremely high reliability and availability
 One, Two, or Three units per rack

IBM Integrated Analytics System Configurations
M4001-003
1/3 Rack
M4001-006
2/3 Rack
M4001-010
Full Rack
Servers 3 5 7
Cores 72 120 168
Memory 1.5 TB 2.5 TB 3.5 TB
User capacity
(Assumes 4x
compression)
64 TB 128 TB 192 TB
IBM Power 8 S822L 24 core server
3.42GHz, IBM FlashSystem 900
Mellanox 10G Ethernet switches

Integrated Analytics System (1/3 rack) - Docker container
 One container per physical server
 Head Node vs Data Node
 3 sizes:
 1/3 Rack (3 servers)
 2/3 Rack (5 servers)
 Full Rack (7 servers)
Head node (node0101) node0102 node0103
 LDAP server and Accelerator server is active on one
node only at any given time

Scaling up/out with Accelerator on Z versus deployment on IIAS
CP
CP
CP
…4-35IFLs…
MLN
MLN
MLN
MLN
MLN
MLN
IDAA on Z: One dashDB Node dashDB Head Node
MLN
MLN
MLN
CP
CP
CP
dashDB Node 2
MLN
MLN
MLN
MLN
CP
CP
CP
dashDB Node 3
MLN
MLN
MLN
MLN
CP
CP
CP
…24CPcores…
…24CPcores…
…24CPcores…
IDAA on IIAS: 3 dashDB Nodes
Cluster Filesystem

Internal Network-Architecture
 Fully redundant topology
 Active-active technology
 Scales to 8 Racks
 Each node uses 4x 10GbE ports to form a
single bonded interface. (Linux mode-4 LACP
bonding)
1
0 fbond
Mlnx 2410 Switch
“fabsw01a”
1
0
1
0
1
0
node0102
Mlnx 2410 Switch
“fabsw01b”
1
0 fbond
1
0
1
0
1
0
node0103
1
0 fbond
1
0
1
0
1
0
node0101
1
0
1
0
1
0
1
0
node0105
fbond 1
0
1
0
1
0
1
0
node0107
fbond1
0
1
0
1
0
1
0
node0106
fbond
2x
40
1
0 fbond
1
0
1
0
1
0
node0104
 Fully redundant network using 1 GbE
 Active-active dual star topology
 Scales to 8 Racks
 Allows resilient management and monitoring
of all rack components
Management Network
Mlnx 8831 Switch
“mgtsw01a”
Mlnx 8831 Switch
“mgtsw01b”
1
mbond
1
node0101
Fab
Switch
1
mbond
1
node0102
1
mbond
1
node0103
1
mbond
1
node0104
1
mbond
1
node0105
1
mbond
1
node0106
1
mbond
1
node0107
1 1
TMS900
1 1
V5000
1 1
Term
Svr1 1
FC
Switch1
PDUs
1
 Fully redundant 16G FC
 Forms a Per Rack SAN and
defines a “Fault Domain”
 Allows high speed multi-pathed
access to all storage elements
from all processing nodes within
the “Fault Domain”
Fibre Channel SAN
Data Fabric Network

High Availability – Hardware Elements
Robust Hardware Elements
• Power8
• IBM Flash System 900
• Completely resilient storage subsystem for the appliance
• two load sharing power supplies, redundant fans, and two separate storage controllers
• RAID5 layout across Flash elements within each module, and then again RAID5 layout
across the Flash modules
Redundant Hardware Elements
• Data Fabric, Management Network and Storage Fibre Channel Network
• pairs of switches are used to provide complete failover redundancy
• Processing Nodes
• Partitions of failed node are reassigned evenly to the surviving nodes within the same rack

High Availability – Node Failover / Failback
 Processing Nodes – Organized into Highly Available clusters to provide continuous operation in the event of failure of one of the nodes
 First attempt to power recycle failed node and make it usable again
 If recycle failed, the data partitions of the failed node are reassigned to the surviving nodes within the same rack
 The system will experience an outage (up to 30 mins if reboot required) while a failover is performed

Encryption of „data at rest“
 Addresses risk of stolen flash cards / flash card replacements (e.g. after failures)
 Integrated Analytics System provides Encryption through IBM Flash 900 built-in encryption
 XTS-AES 256-bit symmetric used built-in
 Used by Accelerator on IAS w/o external key management.
Flash Controller is key-aware so system not protected in case of FlashSystem lost complete.
 Ext. key management leveraging separate key-management-server is on the roadmap
Attention: Data Studio does not display the encryption w/o external key management as encrypted disk although
there’s no data stored in clear. Starting using external key management, encryption will be detected by Data Studio

Db2 Analytics Accelerator Version 7.1,
deployment on IBM Z

• A software appliance running on IBM Z
• Packages the SW stack into an IBM Secure Service Container to deliver a fully self-managed
appliance running in a SSC LPAR that can be deployed in minutes
• Integrates seamlessly into the customer’s IBM Z environment and leverages known LPAR-,
memory and CPU management procedures, including call home support for enterprise hardware
components.
• Uses customer-provided storage to hold the Accelerator-side data
• Scales smoothly with the assignment of available processor cores, initially addressing sizes
comparable up to 1/2 rack PDA (N3001-005) appliances
Deployment on IBM Z

Accelerator on IBM Z – Optimized for Multiple Instances Usage
 Accelerator on IBM Z instance = 1 node = 1 LPAR
 Single instance typically 8 – 35 IFLs
 Single instance typically 0.5 – 8 TB memory
 Different workloads in different instances
 Optimize instance for workload
 Instance size determined by individual workload
requirements
• not “sum of all”
 IFLs may be shared – dev/test environments
 Isolation of independent / competing workloads
DB 1
Accelerator
on IBM Z 1
DB2
z/OS
DB2
z/OS
DB2
z/OS
Setup only symbolic
DB 3
Accelerator
on IBM Z 3
DB 2
Accelerator
on IBM Z 2

Key
advantages
• Out-of-the-box experience
• Workload Optimized System
• Wide set of Analytics use cases
• Proven technology with client
references cross-industry
• Out-of-the-box experience
• Workload Optimized System
• Optimized for True HTAP
• Evolving set of Analytics use
cases
• Download & Go experience
• Homogeneity within IBM Z:
common resources, operation
• Evolving set of Analytics use
cases
Workload
Size
• Very good scale-out • Very good scale-out
• Optimized for very large query
throughput and load
performance
• Good scale-up to full drawer
Db2 Analytics Accelerator
on Pure Data for Analytics
N3001
(Netezza Technology)
Db2 Analytics Accelerator on
Integrated Analytics System
M4001
Db2 Analytics Accelerator
on IBM Z
Key Characteristics of the deployment options

Db2 analytics accelerator on ibm integrated analytics system technical overview

More Related Content

What's hot (20)

Similar to Db2 analytics accelerator on ibm integrated analytics system technical overview (20)

Recently uploaded (20)

Db2 analytics accelerator on ibm integrated analytics system technical overview

Editor's Notes