Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2
0 likes647 views
This document discusses CA Test Data Manager and its capabilities for high performance masking and test data generation for mainframe applications like IMS and DB2. It summarizes how CA Test Data Manager can centrally manage test data across different sources, securely subset and mask data on the mainframe without moving it, and generate synthetic test data scenarios. It also highlights how CA Test Data Manager maintains data integrity and allows test data assets to be controlled and reused across different projects and versions.
Pre-Con Ed: High Performance Masking and Test Data Generation for IMS and DB2
- 2. 2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA. For Informational Purposes Only Terms of this Presentation
- 4. 4 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Test Data on z/OS (the Bad News) § z/OS system of record (70% data still resides here) § Many organisations rely on copies of Production for testing § Secure and reliable in production – what about test? § What and Where is PII data § Legislation – GDPR, demonstrate compliance § Breach Notification – reputation & fines risk § Cost of moving test data to/from platform (the Good News) Best Practice Test Data Management already addresses this impact
- 6. 6 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD CA-Test Data Manager Deployment & Process 1. Centralize 2. Discover 3. Subset 4. Mask 5. Synthesize 6. Find & Reserve 7. Automate Synthetic/Clone Data Masked Subset Created Production Development Web - SOA Test Harness Virtualization Data Masking Subsetting/Data Slicing Data Profiling Coverage Metrics Agile-Designer Test Design Data Design Test Data on Demand UI - Load Test Harness Test Mart Find and Reserve REQUEST/RESPONSE POOLS VERSION CONTROLEXPECTED RESULTS REGRESSION PACKS DATA DRIVEN TEST PACKS TEMPLATES Agile-DataHP ALM/QC Automation Data Visualization Javelin –Data Orchestration
- 7. 7 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD CA-Test Data Manager Deployment & Process Adabas z/Series IMSDB2 VSAM Unisys DMSII LUW Informix MSSQL Oracle Teradata DB2/400 Flat Files (CSV, Excel, XML etc.) i/Series Windows UI ODBC/JDBC COBOL/JCL JAVA SQL LUW CA – Test Data Manager Register & Profile Define Test Data Engineering Operation Process Run-time Enable as On-Demand Request Test Data On-Demand IMGR & Service Layer Oracle or SQL Server Repository
- 9. 9 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD End-End Integrity § Across multiple source (and platforms!) – z/Series: DB2, IMS, VSAM (KSDS, ESDS, etc.) or flat files (PS) etc. – i/Series – AS400: DB2/400 – Distributed: Oracle, SQL Server, DB2 UDB etc. – File structures: CSV, Excel, XML, SOAP, REST, JSON etc. Maintaining database, business and logical integrity CustID Item# Order Date 07221 80-25442 2015-01-25 07221 80-25442 2015-03-14 CustID Item# Order Date 10002 80-25442 2015-02-15 10002 80-25442 2015-05-19 Data retains structure, context and referential integrity but does not contain “real” information. CustID Name SSN Street 08052 Andrew Chapman 181-23-7171 87 Featherbed Lane 09112 Brian James 112-32-4435 12 Park Lane 08052 Andrew Chapman 181-23-7171 87 Featherbed Lane 07221 Tom Angle 771-11-8972 8 Boardwalk CustID Name SSN Street 10000 Hikaru Sulu 545-95-1245 87 Featherbed Lane 10001 Nyota Uhura 802-41-3411 12 Park Lane 10000 Hikaru Sulu 545-95-1245 87 Featherbed Lane 10002 Pavel Chekov 325-74-6517 8 Boardwalk
- 10. 10 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Project and Version Centralized Control § Re-usable assets in a Test Data Warehouse § Provides a central library of test data engineering operations § Assets can be upgraded across versions (releases) § Regression packs can be stored CA Test Data Manager Test Data Warehouse (Repository)
- 11. 11 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Automatically discover sensitive data stored enterprise-wide § Powerful, mathematically-based algorithms discover and tag all sensitive data, enterprise-wide § Simple, easy to use filtering by tag when selecting columns or tables to mask § Support for every major database type, mainframe platforms and flat files CA Test Data Manager Data Profiler
- 13. 13 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD De-sensitize millions of rows of referentially intact data in minutes CA Test Data Manager Fast Data Masker These include substitution, randomization, hashing and seed data. Once created, masking rules can be stored and re-used from a central repository. GT Fast Data Masker will only display suitable masking routines, based on the selected column and tables Deterministic masking functions and built-in cross-referencing ensure consistency. The referentially intact, realistic data, can be injected into multiple systems at once Number of Rows Time to Mask 201,722,392 9 minutes, 42 seconds 453,877,152 11 minutes, 22 seconds 768,088,071 7 minutes, 57 seconds 17,422,541 1 minute 46,579,485 1 minute, 25 seconds 1,759,612 13 seconds 47, 895 5 seconds GT Fast Data Masker uses native database scripts to produce the highest possible performance when masking Oracle, MS Server, Teradata and mainframe platforms § Mask millions of rows of data in minutes using native masking engines § Over 80 built-in data masking functions § Replacements with realistic, referentially intact, applied to multiple systems § Demonstrate Compliance with the EU GDPR, GLBA, HIPAA, PCI DSS, PIPEDA, and more
- 14. 14 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Generate Required Data Scenarios CA Test Data Manager Synthetic Data Generation This means that new data can efficiently and quickly injected into existing databases for exactly the requirements you need § Extensive algorithms for Synthetic data applied in a Data Painter style § Use Symbolic values, combine functions, include conditional values and mine data from other sources into the new data § Publish (inject/load) single data object as multiple/varied new rows in existing database § Store data objects centrally, as re-usable assets in a Test Data Warehouse § Data can be re-used across versions and releases, with versioned regression packs
- 16. 16 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Combine TDM Operations to Provide Comprehensive Set of Data Referential Integrity Profiling PII Discovery Native DB Engines Seed Replacement Audit CA Test Data Manager 1 2 3 Secure Data Subsets Test/Dev Environments Gold Copy Subset Seed Tables Built-in Masking Functions Cross Referencing Masked Generated Scenarios Find & Reserve Data Synthetic Content Synthetic Content Production data sources and files XML Files Excel Files SQL Files CSV Files Fixed Definition Files HTML Files VSAM/ISAMSwift TXT Files Test Data Warehouse
- 17. 17 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Avoid costly mistakes when managing z/OS Test Data Avoid use of raw production data is a mandatory operation for modern organizations. The average cost per incident of data breach worldwide is $3.5m. In the US, the risk was as high as $5.85m1. In as many as 59% of these incidents, the risk could have been mitigated by implementing Test Data Management best practices 1 Ponemon Institute, 2014 CA TDM allows users to apply Test Data Management practices directly on the mainframe reducing the reliance on slow, expensive Extract, Transform and reload technologies to ensure compliance and take advantage of fit-for-purpose test data. Processing data directly on the mainframe can provide teams with millions of rows of high quality, meaningful, referentially intact data, in a short period of time and at less risk to quality!
- 18. 18 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Recommended Sessions SESSION # TITLE DATE/TIME MFX58EC CA Testing Tools Educational Session (Part 3 of 3): CA Mainframe Application Tuner Demo and What's New 11/15/2016 at 2:00 pm MFX112S Mainframe Application Development Portfolio Review 11/16/2016 at 12:45 pm MFT148S Achieve DevOps on the Mainframe for Faster Time to Market 11/16/2016 at 4:30 pm
- 21. 21 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Thank you. Stay connected at communities.ca.com
- 22. @CAWORLD #CAWORLD © 2016 CA. All RIGHTS RESERVED.22 @CAWORLD #CAWORLD Mainframe and Workload Automation For more information please visit ca.com/appdev Prashant Pandey Sr Principal Consultant, Presales prashant.pandey@ca.com