Stop Bad Data in its OLTP Tracks

Stop Bad Data in its
OLTP Tracks
Denise McInerney
Silicon Valley SQL Server Users Group
June 19, 2012
Copyright © 2012 Denise McInerney

Agenda
Topics
 What is “bad” data?
 Table design
 NULL pitfalls
 Constraints: friend or foe?
 Hardcoding
 Transactions
 Garbage in
 Testing
 Change Management & Controls
June 19, 2012 2Copyright © 2012 Denise McInerney

Who am I?
• SQL Server DBA since 1998
 Web-based OLTP applications
 Focus on design and
performance tuning
• Staff DBA Intuit
• Director, PASS Board
• Virtual Chapters
• Women in Tech VC
• Contact me:
 denise.mcinerney@sqlpass.org
 denise_mcinerney@intuit.com
 @denisemc06
Copyright © 2012 Denise McInerney 3
June 19, 2012

My Framework
 DBA = data steward
 Backups, security—what about quality?
 DBA <> Developer
 Don‟t count on ETL for everything

Define “bad” data
 Inaccurate
 Missing
 Misleading
 Causes bugs
 Breaks reports
 Can compromise security
 Impacts customers
 Impacts business

Design
 Have one—even in an “agile” shop
 It‟s a system, not a collection of tables
 Don‟t be lazy
 Bad design invites bad data

Design
June 19, 2012 Copyright © 2012 Denise McInerney 7

Bad Design Examples

Pitfalls of NULL
 What is NULL?
 Missing data
 Unknown in comparisons
 NULL <> „‟ or “”
 NULL <> 0
 NULL can be valid
 But application can‟t handle it

NULL Examples

Constraints: Your Frenemies
 CHECK
 INSERT/UPDATE fails if condition evaluates to
FALSE
 Comparisons to NULL result in unknown
 Can use to enforce logic, business rules
 DEFAULT
 Can prevent some bugs
 Won‟t solve all your problems
 Can be “forgotten”
 100% of records have default value

Constraint Examples

Hardcoding

Transactions
 Understand them
 Don‟t assume others understand them
 BEGIN TRAN…COMMIT TRAN is not
enough
 Failed INSERT…the silent killer

Transactions

Garbage In
 Many ways things go wrong
 Concatenation
 Recycled identifiers
 Duplicates
 Honor the schema
 Be consistent
 Fix it on the way in

Garbage In

Testing
 Database is not exempt
 Not just the happy path
 Look at the data!
 People
 Machines

Change Control
 Lots of ways data gets IN
 Code—application & SQL
 Version control & Management
 Scripts
 GUI (!)
 People
 Everyone takes short cuts
 Every check-in potential data bug

There is no “I” in “Bad Data”

Pessimism

Vigilance

Summary
 DBA‟s job to guard data quality
 Design, Plan, Process, Controls
 Engage Others
 Keep your guard up

Q & A

Stop Bad Data in its OLTP Tracks

More Related Content

Viewers also liked (14)

Similar to Stop Bad Data in its OLTP Tracks (20)

Recently uploaded (20)

Stop Bad Data in its OLTP Tracks