The Importance of Data for DevOps: How TCF Bank Meets Test Data Challenges

1
The Importance of Data for DevOps
How TCF Banks Meets Test Data Challenges
Irene Ford, Product Manager, Compuware
Lynn Farley, VP, Digital Center of Excellence, TCF Bank
Bismarck Herrera, Senior Data Privacy Engineer, TCF Bank
July 2018

2
Irene Ford
Product Manager
Compuware
Bismarck Herrera
Senior Data Privacy Engineer
TCF Bank
Lynn Farley
VP, Digital Center of Excellence
TCF Bank

3
• Data management
– Support DevOps
– Provide tools to easily manage
data across enterprise
– Provide tools to implement
Agile development
– Help developers, testers
and analysts understand
data and relationships
– Automated testing
DevOps: Deliver more frequent, timely
and higher quality products and services
The Role of Data in DevOps

4
Test Data Management in DevOps
Quality
Produce higher
quality apps
with accurate,
relationally intact
test data
Velocity
Ensure developers/
testers can quickly
understand, access,
extract and compare
cross-platform data
Efficiency
Provide consistent
approach for
working with both
mainframe and
non-mainframe data
Privacy
Enforce privacy
regulations
by protecting
sensitive data from
central resource

5
Continuous Testing
Prepare Test Data within the DevOps Toolchain
Obtain
test data
from production
Mask
test data
Load Gold Copy
test data
Subset for
dev and test
Subset for
integration test
Subset for
acceptance test
Production
Mask Data Before Leaving Production Subset Based on Environment
Test

Who We Are
• A multiplatform IT team with complex integrated applications supported by product development teams
Retail / Consumer
2018 Product Taxonomy
Enterprise services
Mkt &
Customer experience
Digital Banking Account Opening
DotCom
Direct BankCustomer Servicing
Retail Operations
Retail Lending
Wholesale / B2B
Digital Customer Experience (Portal)
Servicing & Customer Care
Originations & Onboarding
Business Intelligence / Data
Enterprise
Operations
Human
Capital
Mngt
Money
Movement
Deposits
Risk &
Financial
Crimes
Prevention
Data &
Analytics
Digital
Workplace
Oracle
Finance IT Hosting
IT
Connectivity
IT End User
Computing
IT
Operations
& Support
IT
Integration
Information
Security

Our Goal as an IT Organization

TDP Mission
To implement efficient data privacy processes (obfuscation, exception reporting, sub-setting) to
satisfy product teams’ needs for quality data for Agile implementations and to ensure that
production data doesn’t leak into non-production environments.
TDP Goals
Build out obfuscation processes for all
data sources (that contain PII)
associated with each product
Leverage in-house automation tools to provide
developers with self-service environment refreshes
Identify all testing scenarios to allow for sub-setting
Implement a more robust exception reporting process
for non-prod environments
Short Term Long Term

TDP Opportunities
GETTING TEST DATA
TO TEAMS
• Complex environments
and relationships
• Multiple data sources
• PII spread across
systems
• Different types of data
LDM
MTG File
files
Trial.txt
Payoffs.txt
LoanBalance.csv
No NPI
data
Lending
System
files
Crystal
Reports
Loan origination
system
(Distributed)
Sort file
(from vendor)
TCF.BD.MTG.EXTRA
CT.SORT
(converted to MTG
legacy format)
Finalist
(fix Addresses)
MTG File
ZIP4 file
MF
TCF.BD.DLS.Loan To
Value.ZIP4(+0)
Files
(from 3 vendors)
FAEX SPEC
FOR INITIAL LOAD:
INITIAL_LOAD.csv
Originated
Loans
*Data is
already
obfuscated
*There’s no
NPI in Non-
Production
environments
Lending
System 8
(Mainframe)
MAINFRAME DATASETS
(PRODUCTION)
MTG
Lending DB
Lending System
Lending System
CHGOFF
CQ
LDS
Core tables
*Schemas 1-7
are part of the
Argo Loan app.
Not in use, no
NPIHistory tables
Lending
System
files
MF JOB
Credit
Atribs
Transactions
ChargeOffs
Loan
origination
Vendor .txt files
ChargeOff Q
Distributed
FCS TEST GL
(Mainframe)
LENDING DATABASE
Loan To ValueLoan To Value
Servicing
Distributed
Cognos
Reports
Master files
by source
Loan File
GL File
Retail Lending
Collateral history
*No finalist applied to addresses on these files
LSBO UI
(Distributed)
Nightly (payments, etc)
SAR
Reports
Deposits/ODP
(Mainframe)
DB
SEND.FTP
External Vendor
files
(from vendor)
ODP LOAN
BALANCE
PRODUCTION
(Mainframe)
Expanded
file
Mainframe dataset
(converted from | delimited to fixed
length)
Mainframe
Jobs
Daily
Weekly
Daily & Weekly
Extract files
(Pipe delimited)
TCF.OD.MTG
NPI INPUTS
Files & tables
1
2
2
3
4
6
7
DB SCHEMA
Application
not in QA
DB not
in QA
File with
no NPI
QA
Application
Address
must be
in sync
Valid address
U.S. Post Office
*80+- tables with no NPI to move
COLLATERAL
LSBO
Lending accounts
Property & servicing tables
Tables with NPI
30+ Tables with NPI
DM DB
ADDRESS
ADDRESS_ID_LOOKUP
CUSTOMER
CUSTOMER_ID_LOOKUP
REAL_ESTATE
LEGEND
Monthly Flow Dependency
Daily Flow Dependency
Existing Data Flow
File with
NPI
Production DB
File
Secure
Location
Table with NPI
New Data Flow
Credit &
Allowance
Text file
COGNOS
reports
Make Legacy Apps/
DBs Whole
[Details TBD]
Obfuscation read
Obfuscation write
NPI
SERVER
EXTRACT
file
Core data
Outside Vendor
CHANGES
5
*Obfuscate test file only
*One time execution
# Executed by scheduler
#
Vendor files
Vendor .txt files
Daily
Weekly
Monthly
8 New NPI sources
PRODUCTION
Pipeline
Borrower
Pipeline
Main
Credit
Bureau.csv
External
Vendor SIT

TDP Opportunities
PRIVATIZING DATA
• Keep data attributes
• Consider business rules
• Preserve data integrity
• Identify integration
points
• Coordinate
data refreshes
• Choose the right tools
LDM
Vendor
files
Trial.txt
Payoffs.txt
LoanBalance.csv
No NPI
data
Lending
System
files
Credit
Bureau.csv
Crystal
Reports
Loan origination
system
(Distributed)
Sort file
(from vendor)
TCF.BD.MTG
FILE.EXTRACT.SORT
(converted to MTG
FILE legacy format)
Finalist
(fix Addresses)
MTG File
ZIP4 file
MF
TCF.BD.DLS.Loan To
Value.ZIP4(+0)
Files
(from 3 vendors)
FAEX SPEC
FOR INITIAL LOAD:
INITIAL_LOAD.csv
Originated
Loans
*Data is
already
obfuscated
*There’s no
NPI in Non-
Production
environments
Lending
System 8
(Mainframe)
MAINFRAME DATASETS
(PRODUCTION)
MTG
Lending DB
Lending System
Lending System
CHGOFF
CQ
LDS
Core tables
*Schemas 1-7
are part of the
Argo Loan app.
Not in use, no
NPIHistory tables
Lending
System
filesFILE-AID DS
OBFUSCATED
MAINFRAME
DATASETS
QA & TEST
MF JOB
Credit
Atribs
Transactions
ChargeOffs
FILE-AID EX
Ln Orig
Vendor .txt files
Ln Orig
6 hfs_Lending Systemnote
obfuscated files
FILE-AID EX
ChargeOff Q
Distributed
FCS TEST GL
(Mainframe)
FILE-AID EX
Trial.txt
Payoffs.txt
LoanBalanc
e.csv
LENDING DATABASE
Loan To ValueLoan To Value
Servicing
Distributed
Cognos
Reports
(from vendor)
ODP LOAN
BALANCE
Master files
by source
Loan File
GL File
Retail Lending
OBFUSCATION
PROCESS
Collateral history
*No finalist applied to addresses on these files
LSBO UI
(Distributed)
Nightly (payments, etc)
SAR
Reports
Deposits/ODP
(Mainframe)
DB
SEND.FTP
External Vendor
files
(from vendor)
External Vendor
files
FILE-AID EX
ODP LOAN
BALANCE
FILE-AID EX
PRODUCTION
(Mainframe)
Vendor
file
Expanded
file
Mainframe dataset
(converted from | delimited to fixed
length)
Mainframe
Jobs
Daily
Weekly
Daily & Weekly
Extract files
(Pipe delimited)
TCF.OD.MTG
FILE
TCF.OD.MTG
FILE
FILE-AID DS
*Obfuscate once; static file
NPI INPUTS
Files & tables
1
2
2
3
4
6
7
DB SCHEMA
Application
not in QA
DB not
in QA
Obfuscated
file
File with
no NPI
Obfuscation
process
QA
Application
Data
Solutions
Jobs
DLSPD01O
Data
Solutions job
Q_External Vendor_ODP_LOAN_BALANCE
ConverterPro
specifications
FAEX
ConverterPro specifications
Address
must be
in sync
Valid address
U.S. Post Office
NPI
SERVER
*80+- tables with no NPI to move
COLLATERAL
LSBO
Lending accounts
Property & servicing tables
Tables with NPI
8
30+ Tables with NPI
DM DB
ADDRESS
ADDRESS_ID_LOOKUP
CUSTOMER
CUSTOMER_ID_LOOKUP
REAL_ESTATE
FILE-AID EX
9
DM DB.LDM
(Production)
Lending DB
(Production)
NEW
*Table to table
FILE-AID EX
Refresh
COPY
*Table to table
LEGEND
Monthly Flow Dependency
Daily Flow Dependency
Existing Data Flow
File with
NPI
Production DB
File
Secure
Location
Obfuscated
DB
Obfuscation
job to fix
Table with NPI
New Data Flow
Credit &
Allowance
Text file
COGNOS
reports
Make Legacy Apps/
DBs Whole
[Details TBD]
Obfuscation read
Obfuscation write
NPI
SERVER
EXTRACT
file
*Create process
to send obfuscated
file to MF
Core data
Outside Vendor
CHANGES
5
*Obfuscate test file only
*One time execution
Obfuscated .txt files
# Executed by scheduler
#
Vendor files
Vendor .txt files
Daily
Weekly
Monthly
Vendor .txt files
Dayly
Weekly
Monthly
8 New NPI sources
DEV & QA
Pipeline
Borrower
FILE-AID EX
Pipeline
Borrower
Pipeline
Main
Pipeline
Main
5A
Credit
Bureau.csv
FILE-AID EX
10
External
Vendor SIT

TDP Opportunities
GOING AGILE
• Multiple sprints running
simultaneously
• Higher demand
for testing data
• Short sprints
• Same quality data
• Less time to
troubleshoot issues
Product Team 1
Test load process
from vendor file
Product Team 2
Unit testing
Integration testing
Product Team 4
Performance testing
Product Team 3
BUILDING
PLANNING
ANALYSIS
DESIGN
SPRINT
TESTING
&
INTEGRATION

TDP Opportunities
BUILDING THE RIGHT TDP
ENVIRONMENT FOR YOUR SHOP
• PII is locked down;
no easy access
• Restrictions to move
data from Prod to Test
• Credentials and permissions
• Shared resources
• TDP tools maintenance
• TDP tools upgrades

TDP Opportunities
MAKING SURE THE DATA STAYS CLEAN IN TEST
• Users introduce PII in lower environments
• Production files are loaded in testing regions
• System malfunction introduces PII in non-production environments
• Users file exceptions to use Production data

TDP Actions to Address Opportunities
• Updated TDP roadmap
• Privacy by design
• Exhaustive data analysis
• Built new environment
• Upgraded tools
• Optimized migration test/production process
• On Demand Exception Reporting to identify
PII in non-prod environments

TDP Actions to Address Opportunities – In Progress

Conclusion
• Data is important in DevOps
• Prod-like data in test increases reliability
and reduces time to delivery
• Reduces bugs in production
• Partnership between TCF and Compuware
• Responsive and reactive to software issues
• Supportive in helping us accomplish
our vision for self-service environment
refreshes of obfuscated data

18
Questions and Next Steps
Continue the Conversation
compuware.com/contact
Learn More About Compuware
Test Data Privacy (Webpage)
compuware.com/test-data-management
Get More Information
on Topaz for Enterprise Data
(Short Video)
compuware.com/video/ted

The Importance of Data for DevOps: How TCF Bank Meets Test Data Challenges

More Related Content

Similar to The Importance of Data for DevOps: How TCF Bank Meets Test Data Challenges (20)

More from Compuware (20)

Recently uploaded (20)

The Importance of Data for DevOps: How TCF Bank Meets Test Data Challenges