Old Dogs, New Tricks: Big Data from and for Mainframe IT

Ed Hallock – Director of Product Marketing
and Management
Old Dogs, New Tricks: Big Data from and for
Mainframe IT

Housekeeping
• Webcast Audio:
– Today’s webcast audio is streamed through your computer speakers.
– If you need technical assistance with the web interface or audio, please reach out
to us using the chat window.
• Questions Welcome:
– Submit your questions at any time during the presentation using the chat
window.
– We will answer them during our Q&A session following the presentations.
• Recording and Slides:
– This webcast is being recorded. You will receive an email following the webcast
with a link to download both the recording and the slides.
© 2016 Syncsort Incorporated

Today’s Presenter
Syncsort Confidential and Proprietary - do not copy or distribute
Ed Hallock is a highly experienced Information Technology Professional
with a broad experience base in software product development, support,
product management, marketing, and business development. In his
diverse career Ed has benefited from working for some of the largest
independent software vendors, in a variety of roles, providing enterprise
solutions to Global 1000 corporations. Ed has extensive experience in
performance and availability management for systems and applications.
He holds a bachelor’s degree in Computer Science from Montclair State
University in Upper Montclair, New Jersey and has presented at numerous
industry events as well as corporate related conferences and seminars.

Agenda
Big Iron to Big Data Analytics Challenge
State of the mainframe
IT Operations Analytics
Security Information and Event Management
IT Service Intelligence
Summary and Q&A

Big Iron to Big Data Analytics Challenge
So many data sources
– SMF, Syslog, Log4j web and application logs, RMF, RACF,
USS files and standard datasets
Volume of data
– Millions of SMF records generated daily
Format of data
– Complex data structures (SMF) with headers, product
sections, data sections, variable length and self-describing
– EBCDIC not recognized outside of the mainframe world
– Binary flags and fields
Difficult to get the information in a timely manner
– Not real-time, typically have to wait overnight for an
offload

What Has Been Done in the Past?
Performance Monitors
– Proactively analyze and manage z/OS
operating systems, databases other z/OS sub-
systems for optimal performance
– Very good at detecting bottlenecks and other
potential performance problems in z/OS,
CICS, IMS, DB2, MQ, Storage, etc.
– Most include historical reporting and trending
facilities but that is typically limited to a
subset of the data that the monitor collects
Capacity Planning Tools
– Next day, next week, next month reporting of
offloaded SMF data
Event Management Systems
– Alert management

Challenges with these Legacy Technologies
Tend to have fixed displays with little room for
customization on how an end-user can see
data provided
The interface(s) to these products have
traditionally been closed and proprietary
Limited view into security issues and threats
Limited ability to monitor business services
and provide service-level intelligence
They typically have a silo approach: a monitor for DB2, another monitor for
CICS, etc. without any real correlation between the different pieces
Require Subject Matter Experts (SMEs) with in depth technical knowledge of
z/OS and its sub-systems in order to effectively use the products
Most have evolved into very complex and resource intensive solutions in an
attempt to cover ever aspect of the systems they monitor

Big Iron Trends to Watch for in 2017: Big data analytics for operational intelligence,
security, and compliance will continue to grow and emerge as a critical project in organizations.
9Syncsort Confidential and Proprietary - do not copy or distribute

Big Iron Trends to Watch for in 2017: Increased interest for real-time access to mainframe
machine data (SMF, RMF, log data, etc.) for business analytics

Big Iron Trends to Watch for in 2017: Mainframe-based tools and batch processes will
have to yield ground to new technologies including Hadoop, Spark, and Splunk for big data analytics.

Polling Question #1
What analytics platforms are you using today for z/OS IT operational intelligence:
 Splunk
 Hadoop
 ELK (Elastic Stack)
 Spark
 Custom/Home Grown solution
 None
12

What is Needed?
High performance, low-cost, platform for collecting critical system
information in real-time
Normalization of the z/OS data so it can be used off platform
analytics engines
Full analytics, visualization, and customization with no limitations
on what can be viewed
Ability to easily combine information from different data sources
and systems
Address the SME challenge: use by network managers, security
analysts, application analysts, enterprise architects without
requiring mainframe access or expertise

What is Needed?
It’s no longer about determining problems and preventing outages - unforced
IPL’s are a rarity
Need deeper analytic capabilities that includes integration across technology
silos
– IT Operational Analytics (ITOA)
• Capacity optimization vs. Capacity planning -- getting the most of existing
capacity vs. determining when to buy more
• Achieving operational efficiency
– Security Information and Event Management (SIEM)
• Security monitoring Threat detection
• Audit and regulatory compliance
– IT Service Intelligence (ITSI)
• Understanding IT component relationships and their impact on service delivery
• Business service responsiveness

IT OPERATIONS ANALYTICS (ITOA)

What is ITOA?
IT Operations Analytics (ITOA): an approach to IT operational data that allows for
better understanding and enabling better decisions about managing the IT
environment.
Applies Big Data principles to the IT environment providing a broader context—and
clearer operational intelligence—about what's happening.
Bigger picture of what's happening in the environment and make better decisions to
take control of the IT infrastructure.

ITOA: MSU 4-Hour RA with GP and zIIP Utilization by TOD
MSU 4-Hour
Rolling Avg. zIIP Utilization
by LPAR
CPU Utilization
by LPAR

ITOA: Analysis of SORT CPU Consumption
SORT CPU
Utilization by
TOD
SORT zIIP
Offload
Potential
SORT CPU
Utilization
by Date
SORT CPU
Utilization
by Day of
Week

ITOA: Abend Analysis
Abends Over Time
By LPAR & Abend Code
Prod vs. Test

ITOA: CICS Region Health Check
Syncsort Confidential and Proprietary - do not copy or distribute 20
Transaction Response Time Transaction Rates
Dispatch Time

SECURITY INFORMATION AND EVENT
MANAGEMENT(SIEM)

What is SIEM?
Security Information and Event Management (SIEM) technology aggregates and
provides real-time analysis of security alerts.
– Event data is produced by security devices, network infrastructures, systems
and applications. The primary source is log data (i.e., SMF RACF records).
Analyze security event data in real time for internal and external threat detection
and management.
– Every organization fears potential hacks and data loss.
Collect, store, analyze and report on log data for incident response, forensics and
regulatory compliance.
– Meeting audit requirements is a key initiative in many vertical industries
SIEM products are gaining additional analytics capabilities around user behavior
analytics (UBA) – understanding user behavior and how it might impact security

Examples of SIEM Use Cases on z/OS
Detect Data Movements
– Inbound/Outbound FTP
Dataset access operations
– Determine potential security threats based on unauthorized access attempts
– Ensure only authorized users are accessing critical datasets
Privileged/non-privileged User Activity Monitoring
– Unusual behavior pattern – off hours connections
– High number of invalid logon attempts
Attack Detection
– Intrusion, Scans, Floods
Authentication Anomalies
– Entered the building at 08:30 but logged on from another country at 09:00
Network Traffic Analysis
– High data volumes from a device/server

SIEM: TSO Account & FTP Activity
Job Initiations
TSO Account Activity
TSO Lockouts
FTP Session Activity
FTP Transfer Activity

SIEM: Intrusion Detection & TCP/IP Traffic
Intrusion Detection showing Port Scans and
Denial of Service Attacks
TCP/IP Network Traffic

SIEM: Dataset Access Analysis
Access by type for critical datasets by
user name

SIEM: RACF Violations and Message Trends
RACF Violations by type RACF Violations by user
Trend message volumes today vs. same time last week and 2 weeks ago

Polling Question #2
What Security Information and Event Management (SIEM) platform is in use within
your Enterprise:
 IBM zSecure/QRadar
 Correlog
 Splunk Enterprise Security
 HP Arcsight
 Logrythm
 Other

IT SERVICE INTELLIGENCE (ITSI)

What is ITSI?
Delivers a central, unified view of critical IT services for powerful, data-driven
monitoring
Maps critical services with KPIs to easily pinpoint what matters most
Uses machine learning to detect patterns, dynamically adapt thresholds, highlight
anomalies and pinpoint areas of impact
Provides business and service context to prioritize incident investigation and triage
Supports drill downs to profile an entity and rapidly troubleshoot outages and
service degradations

Providing z/OS Metrics & Analysis to IT Service Intelligence
3 Levels of Information Needed
for a complete picture
Overall Mainframe Central Processor Complex
LPAR Logical Partition (virtual machine equivalent)
Software Components
 CICS online transaction processing
 DB2 database, typically used with CICS

Splunk Service Analyzer --
A Unified View of Critical Services
32
Problem with
Online Banking

Service View Across All 3 Levels
33
Critical services
CEC (Central
Electronic
Complex)
LPARs (logical
partitions)
CEC is Fine
LPARs OK?
Online Banking has DB2
problems

Polling Question #3
What analytics platforms are you considering or evaluating to use for z/OS IT
operational intelligence:
 Splunk
 Hadoop
 ELK (Elastic Stack)
 Spark
 Custom/Home Grown solution
 Other
35

SUMMARY
36

Summary: Where We Need to Go
Offload of z/OS operational and security data for deeper analytics
Adopt analytics platforms to address challenges
– Splunk, Hadoop, Spark, ELK, etc.
Have a 1-stop shop(platform) to address all the needs across the organization
– ITOA, SIEM, ITSI
Close the mainframe knowledge gap
– Standard browser interfaces vs. proprietary UI’s
– Normalization of mainframe specific data types(ex: EBCDIC to ASCII)
– Data models to simplify understanding the data
– Simplified customization to present desired metrics

Critical Mainframe Data 
Normalized and Streamed to Splunk with Ironstream®
Log4jFile
Loa
d
SYSLO
GSYSLOG
Dlogs
securit
y
SMF
50+
types
RMF
Up to
50,000
values
DB
2
SYSOU
T
Live/Stored
SPOOL
Data
Alerts
Network
Component
s
Ironstream
API
Application Data
Assembler
C
COBOL
REXX
USS

Ironstream Apps Are Now On Splunk App Store (splunkbase)
https://splunkbase.splunk.com/
 Search Syncsort

Get Ironstream® for SYSLOG for free
http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition

Industry Leader in Mainframe
Software Products

Old Dogs, New Tricks: Big Data from and for Mainframe IT

More Related Content

What's hot (20)

Similar to Old Dogs, New Tricks: Big Data from and for Mainframe IT (20)

More from Precisely (20)

Recently uploaded (20)

Old Dogs, New Tricks: Big Data from and for Mainframe IT