Loshin operationalizingdatagovernance
1 like342 views
The document discusses operationalizing data governance through data quality control. It describes linking business value drivers to data expectations and objectives through data policies, processes, and oversight. It also discusses assessing data quality, defining rules and measures, monitoring data, and integrating quality reporting into the governance framework. The goal is to ensure data accuracy, completeness, and consistency to maximize business value.
Loshin operationalizingdatagovernance
- 1. Operationalizing Data Governance through Data Quality Control David Loshin Knowledge Integrity, Inc. www.knowledge-integrity.com 1© 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350
- 2. Linking Organization to Practice p Mapping business value drivers to data expectations and objectives, granting oversight and accountability, and verifying performance of compliance with corporate information policies n Processes prescribed for operations, n Procedures for day-to-day observance n Oversight for verifying compliance © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 2 Accountability Data Policies Processes & Best Practices Information Standards Business Policy Roles and responsibilities Program management Data policies and standards Business intelligence Business terminology Data quality Auditability
- 3. Business Value and Data Dependence © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 3 Expenses Risk Management Revenue Customer Experience Performance p Business policies, corporate mission, and strategic performance objectives can be translated into dimensions of value p These criteria are used for prioritizing effort in relation to maximizing value p Data governance helps establish the relationship between value drivers and information utility
- 4. Sources of Information Policy © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 4 Expenses Risk Management Revenue Customer Experience Performance • Customer lifetime value analysis • Voice of the customer • Satisfaction surveys • Asset productivity analysis • Human capital performance • Regulations • Operational risk • Market factors • Customer acquisition and retention • Investment opportunities • Product performance • Spend analysis • Commodity risk • Cost management
- 5. Data Management Challenges Data Use and ReUse Data Requirements Reinterpreting Semantics Measurement Triage and Remediation Impact Assessment © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 5
- 6. The Data Policy Lifecycle: Actualizing Governance p Refinement of business requirements for facets of information utility p Specification for data quality measures and level of acceptability p Determination of functional requirements to facilitate continuous compliance © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 6 Determination of need Drafting a Policy Policy Review & Approval Design & Development Marketing Deployment
- 7. Information Policies and Data Governance Integrated Data Governance Manage User Requirements Data Discovery Shared Semantics Embedded Validation © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 7 • Business metadata management • Data quality discovery and assessment • Specifying data quality rules • Inspection, monitoring, measurement • Managing data lineage
- 8. Managing the Quality of Business Metadata p Many sources of entity concepts and business terms may conflict with each other p The data governance framework must facilitate the collection, documentation, and harmonization of business terms © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 8 Policies System Docs Processes Models Standards Applications Business Rules Profiling Etc. Entity Concepts Business Terms Definition Contextual Meaning … … Definition Contextual Meaning Definition Contextual Meaning Definition Contextual Meaning
- 9. Data Discovery p Data Discovery enables these types of questions to be answered: n What data sets are available? n What entities are embedded? n What data elements are available? n How is the data accessed? n What are the quality constraints? p The results can be shared via a platform for managing semantic metadata © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 9 DataIntegration
- 10. Attribute First d 4 6 y Last f 6 2 h Street d 4 7 n City a 0 2 o State Value Count A 12000 I 10000 L 7655 X 3208 N 120 M 8 Data Quality Assessment p Analysis of data sets, records, data elements, and data values to n Identify potential anomalies n Determine business impacts n Evaluate dimensions for measurement of data quality © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 10 Analysis
- 11. Data Quality: Expectations, Rules, and Monitoring p Data quality rules can be used to monitor conformance to data policies p Conformance can be measured, thresholded, and reported at each handoff location in the processing stream p Specific failures can generate events as directed by Data Quality Service Level Agreements p Static auditing: measurement applied to a “static” data set n Examples: SQL queries, data profiling tools p Inlined monitoring: measurement performed within a process flow n Example: edit checks, dynamic monitors p All measurements are compared against acceptability thresholds p Acceptability threshold is related to the degree of impact 1111© 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350
- 12. Data Quality Rules: Measures and Thresholds p Provide specific n Measures n Methods of measurement n Units of measures n Levels of acceptability © 2012 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 12
- 13. © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 1313 Data Quality Control p Controls measure observance of data expectations based on information policies and corresponding data rules p Those rules are refined based on an analysis of the data dependencies and defined expectations p Controls are placed at relevant locations within the process stream Producer Process Consumer Process As data is handed off between process tasks, controls validate accuracy, completeness, consistency, timeliness against defined expectations
- 14. Instituting Inspection Using Data Quality Rules p Apply tools and techniques for measuring conformance to data rules (using data profiling and data monitoring tools): © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 14
- 15. Instituting Inspection Using Data Quality Rules p Data quality expectations are inspected and any emerging issues are identified: © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 15
- 16. Instituting Inspection Using Data Quality Rules p Different events can be triggered by a data failure, such as notifications to data stewards: © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 16
- 17. Instituting Inspection Using Data Quality Rules p Or logging the failure in a Data Quality Incident Management System and score card: © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 17
- 18. Instituting Inspection Using Data Quality Rules p Effectiveness demonstrated when: n Control events occur when data failure events take place, n The proper mitigation or remediation actions are performed, n The corrective actions to correct the problem and eliminate its root cause are performed within a reasonable time frame, and n A control event for the same issue is never triggered further downstream p Measurements can be aggregated over time into performance metrics © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 18
- 19. Integrating Data Quality Reporting with Governance © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 19 Processing Stage Processing Stage Processing Stage Processing Stage
- 20. Tools & Processes: Operationalizing Data Governance p Methods and tools for data discovery: profiling data, statistical analysis of values, and model evaluation p Metadata management through a central platform for knowledge capture and communication p End-to-end visibility of lineage for structure, semantics, and use across enterprise p Data Quality assessment p Integrated data quality control p Inspection, monitoring, and reporting © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 20
- 21. Questions and Open Discussion p www.knowledge-integrity.com p If you have questions, comments, or suggestions, please contact me David Loshin 301-754-6350 loshin@knowledge-integrity.com © 2013 Knowledge Integrity, Inc. www.knowledge-integrity.com (301)754-6350 21 www.dataqualitybook.com www.mdmbook.com