Collaborative Metadata Management with David Loshin
0 likes594 views
The document discusses the importance of business-IT collaboration in managing enterprise metadata through consistent definitions and shared understanding of data across different functions. It highlights the challenges of ambiguous terms, the necessity of a centralized metadata repository, and the role of data stewards in harmonizing business terms to ensure semantic consistency. Additionally, the document emphasizes the need for a collaborative platform to facilitate the tracking, governance, and resolution of metadata discrepancies.
Collaborative Metadata Management with David Loshin
- 1. Breaking Down the Fence: Using Business-IT Collaboration to Foster Consistent Enterprise Metadata David Loshin Knowledge Integrity, Inc. www.knowledge-integrity.com 1© 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350
- 2. Islands of Data? p Different business functions have their own views of necessary data p Requirements are based on each function’s particular needs p Data subsystems represent each function’s particular views © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 2 Finance Financial Data Sales Sales Data Marketing Marketing Data Service Service Data H/R H/R Data Transactions Transaction Data Compliance Compliance Data
- 3. Cross-Function Data Sharing p Cross functional applications requires data sharing and exchange p Data concepts may exist in different forms in different applications © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 3 Finance Financial Data Sales Sales Data Marketing Marketing Data Service Service Data H/R H/R Data Transactions Transaction Data Compliance Compliance Data
- 4. The Most Dangerous Question in Data Management p Hundreds have posed it… p Thousands have considered it… p Tens of thousands of hours have been invested in answering… p The most dangerous question in data management: © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 4
- 5. Ambiguity in Contextual Definitions p Example: Uses of the business term “customer” p These are potentially conflicting definitions p Correspondingly, the representations and underlying meanings may differ p Identify business terms and provide a means for n Identifying the authoritative definitions n Resolving similar concepts n Differentiating distinct concepts © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 5 Sales: Someone who pays for our products or services Support: Someone who has a license for use of our product
- 6. The Challenge of Meaning in Different Contexts © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 6 p “Employee hire date” p Any selected definition changes the meaning of the other uses! p Any use may be dependent on contextual meaning The date the employer offered the position The date the prospect accepted the offer The first date the employee came to work The date the employee first provided services for pay The date the rehired employee resumed providing services for pay
- 7. Understanding the Data Lifecycle p Data element use is considered only at design and implementation © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 7 Conception Definition Design Implementation Deployment Utilization Maintenance Repurposing Retention Retirement p A more mature approach looks at data repurposing p Ultimately, all lifecycle events have implications to structure and semantics
- 8. Evolution of the Business Metadata Glossary p Many sources of entity concepts and business terms may conflict with each other p The data steward must facilitate the collection and documentation of business terms p The data steward must also prepare for harmonization of terms © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 8 Policies System Docs Processes Models Standards Applications Business Rules Profiling Etc. Entity Concepts Business Terms Definition Contextual Meaning … … Definition Contextual Meaning Definition Contextual Meaning Definition Contextual Meaning
- 9. Documenting Metadata: Business Term Definitions p Goal: n Identify key business terms n Find authoritative definitions n Document definition(s) and use(s) p Establish a template to capture: n Business term n Type of term (noun, verb, etc.) n Definition n Authority n Usage © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 9 Business Term ID Term Type Definition Authority Usage Biii- HQLA1 High- quality liquid asset Noun, Entity “Assets that can be immediately converted into case at little or no loss of value.” “Basel III: International framework for liquidity risk measurement, standards and monitoring ” http://www.bis.org/ publ/bcbs188.pdf Standard for Liquidity Coverage Ratio (LCR) in “Basel III: International framework for liquidity risk measurement, standards and monitoring ” http://www.bis.org/publ/ bcbs188.pdf
- 10. Evolving a Community of Interest p Different stakeholders have a vested interest in variance in n Naming n Definition n Use of common business terms and entity concepts p Each stakeholder may register his/her interest in the status of particular metadata objects © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 101010 Definition Contextual Meaning … … Definition Contextual Meaning Definition Contextual Meaning Definition Contextual Meaning Business Term
- 11. Defining Business Glossary Terms © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 11 Business Term Use Definition Data Element Concept Data Element Data Element Data Element Concept Data Element Data Element Data Element Use Definition Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain
- 12. Identifying Lineage of Common Business Terms © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 12 Business Term Use Definition Data Element Concept Data Element Data Element Data Element Concept Data Element Data Element Data Element Use Definition Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain Conceptual Domain Value Domain Data Element Type First VARCHAR(25) Middle VARCHAR(25) Last VARCHAR(30) SocialSec CHAR(9) Data Element Type FirstName VARCHAR(35) LastName VARCHAR(40) SSN CHAR(11) Telephone VARCHAR(20)
- 13. Centrally Managing a Business Term Glossary p Common business terms are likely to have many variant definitions p All definitions can be captured in a shared, central repository p Variances can be resolved in two ways: n Consolidation of meaning n Differentiation into distinct concepts © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 13 Data Element Type First VARCHAR(25) Middle VARCHAR(25) Last VARCHAR(30) SocialSec CHAR(9) Data Element Type FirstName VARCHAR(35) LastName VARCHAR(40) SSN CHAR(11) Telephone VARCHAR(20) Social Security Number Identifier Unique number assigned by Social Security Administration
- 14. Centralizing Metadata Across the Organization p Logging business terms and their definitions in a centralized repository provides a single source of information about the chain of definition p As data is shared across functions and among business processes, interested parties can review the variety of definitions © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 14
- 15. Collaborative Metadata Management p The repository becomes the “one-stop shop” for information about n Business terms n Definitions n Conceptual domains n Value domains n Data element concepts n Data elements n Entities p A collaborative platform creates an environment for logging a “vested interest” in the metadata © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 15 Metadata
- 16. Harmonization through Collaboration p The collaborative platform exposes the variances that may impact semantic consistency downstream p Harmonization (via resolution or differentiation) can be facilitated by the virtual workgroup p Other potentially interested parties can be invited to join the conversation © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 16 Social Security Number Identifier Unique number assigned by Social Security Administration Authentication Last four digits of number assigned by the Social Security Administration
- 17. Governance through Collaboration p A collaborative platform allows interested individuals to track those business terms, data element concepts, and reference metadata relevant to their business processes p A comprehensive directory provides visibility to all related concepts p Data policy changes can be governed through the collaborative platform © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 17 Social Security Number Identifier Unique number assigned by Social Security Administration Authentication Last four digits of number assigned by the Social Security Administration Identifier Unique number assigned by the company Customer ID
- 18. Encouraging a Culture of Semantic Harmony p Small variance in definitions in isolated functions become magnified when data is shared across functions p Establish a level playing ground by n Instituting a common business term glossary n Harmonizing business term definitions n Unifying shared reference data into conceptual domains and corresponding value domains n Socializing use of shared metadata n Establishing standards for future development n Integrate methods for monitoring compliance with standards © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 18
- 19. Collaborative Metadata: Considerations p Process n Are there defined practices, disciplines, and training to ensure socialization and integration? p Governance n Does the metadata platform incorporate compliance with defined data policies for oversight of shared metadata? p Comprehensiveness n Can the shared platform capture all aspects of lineage, relationships, business glossary, definitions, etc.? p Ease of Use n The system must be easy to use by consumers who will range from being technically proficient IT staff to informed business users p Maintenance n Is the organization willing to support the ongoing investment in maintaining the collaborative resource? © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 19
- 20. Check Out These Resources! p www.knowledge-integrity.com p www.dataqualitybook.com p If you have questions, comments, or suggestions, please contact me David Loshin 301-754-6350 loshin@knowledge-integrity.com © 2014 Knowledge Integrity, Inc. www.knowledge-integrity.com (301) 754-6350 20