Jarrar: Data Schema Integration
2 likes890 views
The document discusses data schema integration, outlining its challenges, including identifying corresponding concepts across different schemas and resolving conflicts. It proposes a framework for integration involving schema transformation, matching, and conflict resolution, utilizing strategies such as global as view (GAV) and local as view (LAV) for effective mapping. Additionally, it highlights various integration methods, from manual to semi-automatic, and emphasizes the importance of resolving structural, classification, and descriptive conflicts.
Jarrar: Data Schema Integration
- 1. Jarrar © 2013 1 Dr. Mustafa Jarrar University of Birzeit mjarrar@birzeit.edu www.jarrar.info Lecture Notes on Data Schema Integration Birzeit University, Palestine 2013 Data Schema Integration
- 2. Jarrar © 2013 2 Watch this lecture and download the slides from http://jarrar-courses.blogspot.com/2013/11/web-data-management.html
- 3. 3Jarrar © 2013 Employee Region Organization City bornIn/ locatedIn/ /WorksIn locatedIn/ Employee Organization /WorksIn Worker RegionCity bornIn/ locatedIn/ Organization Municipality locatedIn / Schema 1 Schema 2 Schema 3 Data Schema Integration: A simple example In ORM: Integrated schema
- 4. 4Jarrar © 2013 Data Schema Integration: A simple example Source: Carlo Batini Employee Organization Empoloyee City Region Municipality Organization in works Schema 1 Schema 2 Schema 3 Employee City Region Organization works in in Integrated schema In ER: born born in
- 5. 5Jarrar © 2013 Challenges of Data Schema Integration Schema Integration has two major challenges: 1. Identification of all portions of schemas that pertain to the same concept, in such a way to unify such different representations in the global schema. 2. Identification, analysis and resolution of the different types of conflicts (heterogeneities) in different schemas. Source: Carlo Batini
- 6. 6Jarrar © 2013 Framework for Schema Integration Schemas Transformation Schemas Matching Schemas Integration Local Schemas Integrated Schema and mappings Source: Advances in Object-Oriented Data Modeling, M. P. Papazoglou, S. Spaccapietra, Z. Tari (Eds.), The MIT Press, 2000 Integration Rules Transformation Rules Matching Rules
- 7. 7Jarrar © 2013 0. Define the integration strategy If the number of local schemas to be integrated is large, the order of schema integration becomes important. Several strategies can be adopted. Input: n source schemas Output: n source schemas + integration strategies Method used: heuristics Framework for Schema Integration One shot strategy S1 S2 S3 IS Pair at a time strategy S1 S2 S3 IS1 IS2 S4 IS Balanced Strategy S1 S2 S3 IS1 IS2 S4 IS - Efficient integration process - Many correspondences between concepts have to be considered together. - Priority to most relevant and stable schemas. - The integration process is more efficient -e.g.: Production, Marketing, Sales. -To be preferred when the cohesion among schemas is high. …
- 8. 8Jarrar © 2013 Framework for Schema Integration 1. Schema transformation (or Pre-integration) Input: n source schemas Output: n source schemas homogeneized Methods used: Model and Design Homogeneization Reduce model heterogeneities as much as possible to make the sources more suitable for integration. Goal: use a single, common data model and format. Source DBs Homogeneized DBs DW Transformation Integration Source: Stefano Spaccapietra
- 9. 9Jarrar © 2013 Schema Transformation Schema Transformation involves: • Data model homogeneization – Where all data sources are described using the same data model. • Design homogeneization – Enforce standard design rules to reduce the number of structural conflicts (e.g., Normalization: one fact in one place) • Reverse Engineering – Reverse engineer the schema from existing data (such as COBOL files, spreadsheets, legacy relational databases, legacy object- oriented databases).
- 10. 10Jarrar © 2013 Example of Design homogeneization (Normalization) ONE TABLE: R1 (#Student, Name, LastName, #Course, CourseName, Grade, Date) Dependencies: – #Student Name, LastName – #Course CourseName – #Student #Course Grade, Date) Normalized Into 3 Tables: One Fact In One Place: R11 (#Student, Name, LastName) R12 (#Course, CourseName) R13 (#Student, #Course, Grade, Date)
- 11. 11Jarrar © 2013 Example of Reverse Engineering Source: Stefano Spaccapietra
- 12. 12Jarrar © 2013 Schema Matching 2. Schema matching (Correspondences investigation) Input: n source schemas Output: n source schemas + correspondences Method used: techniques to discover correspondences Correspondences relate (schema) elements which describe the same phenomena of the real world. – This step aims at finding and describing all semantic links between elements of the input schemas and the corresponding data. – By doing so, one matches between the schemas to be integrated. – This step fixes the conflicts found in the schema.
- 13. 13Jarrar © 2013 Semantics of Correspondences Correspondences relate (schema) elements which describe the same phenomena of the real world. Source: Stefano Spaccapietra
- 14. 14Jarrar © 2013 Asserting Correspondences Finding matching correspondences is done through the use of a rich language for expressing correspondences (matchings). Example: S1.Person S2.Person, With Corresponding Identifiers: Pin, With Corresponding Property: name Source: Stefano Spaccapietra
- 15. 15Jarrar © 2013 Automated Matching • Fully automated matching is impossible, as a computer process can hardly make ultimate decisions about the semantics of data. • But even partial assistance in discovering of correspondences (to be confirmed or guided by humans) is beneficial, due to the complexity of the task. • All proposed methods rely on some similarity measures that try to evaluate the semantic distance between two descriptions. Some state of the art matching systems Cupid (Microsoft Research, USA) FOAM/QOM (University of Karlsruhe, Germany) OLA (INRIA Rhône-Alpes, France / University of Montreal, Canada) S-Match (University of Trento, Italy) … many others
- 16. 16Jarrar © 2013 Examples of Correspondences Source: Stefano Spaccapietra
- 17. 17Jarrar © 2013 Examples of Correspondences Employee Organization /WorksIn Worker RegionCity bornIn/ locatedIn/ Organization Municipality locatedIn/ Schema 1 Schema 2 Schema 3 Previous example
- 18. 18Jarrar © 2013 Examples of Correspondences Source: Stefano Spaccapietra
- 19. 19Jarrar © 2013 Schema Integration & Mapping Generation 3. Schemas integration and mapping generation Input: n source schemas + correspondences Output: integrated schema + mapping rules btw the integrated schema and input source schemas Method used: New classification of conflicts + Conflict resolution transformations GOAL: Creating an Integrated Schema ( IS ) and the mappings to the local databases. Source: Carlo Batini
- 20. 20Jarrar © 2013 GAV and LAV Integration Research has identified two methods to set up mappings between the integrated schema and the input schemas: (1) GAV (Global As View): proposes to define the integrated schema as a view over input schemas. GAV is usually considered simpler and more efficient for processing queries on the integrated database, but is weaker in supporting evolution of the global system through addition of new sources. (2) LAV (Local As View): proposes to define the local schemas as views over the integrated schema. LAV generates issues of incomplete information, which adds complexity in handling global queries, but it better supports dynamic addition and removal of source.
- 21. 21Jarrar © 2013 Integration Process After we identified the correspondences (in the previous step), we now solve the conflicts: One can distinguish between four types of conflicts: – Structural conflicts – Classification conflicts – Descriptive conflicts – Fragmentation conflicts Examples of conflicts among related object types – different classifications (sets of instances) – different sets of properties – different structures – different coding schemes – …
- 22. 22Jarrar © 2013 Integration Rules Rules defining the strategy to solve conflicts Example rules: – If an class corresponds to an attribute, keep the class – If the population of a class is included in the population of another class, build an is-a hierarchy Integration rules depend on how you want the integrated schema to look like
- 23. 23Jarrar © 2013 Structural Conflicts Different schema element types, e.g.: class, attribute, relationship Library example: – S1 : Book is a class – S2 : books is an attribute of Author Conflict resolution : Choose the less constraining structure – Integrated Schema: Book is a class S1 S2 Source: Stefano Spaccapietra
- 24. 24Jarrar © 2013 Classification Conflicts • Corresponding elements describe different sets of real world objects – S1.Faculty CONTAINS S2.PhD-advisor • Conflict Resolution: – Generalization / Specialization hierarchy – Merging Phd-advisor Faculty Phd-advisor FacultyS1 S2 Faculty
- 25. 25Jarrar © 2013 Descriptive Conflicts Corresponding types have different properties, or corresponding properties are described in different ways Object / Entity / Relationship type: – Naming conflicts : • synonyms Node , Extremity • homonyms Highway (EU) , Highway (USA) – Composition conflicts : different attributes and methods • Employee ( E# , name , address ) • Employee ( E# , position , salary , department )
- 26. 26Jarrar © 2013 Integration Methods: Manual Easy to implement , Flexible BUT time consuming for the DBA a language schemas integrated schema mapping rules DBA First method: manual integration “ do it yourself ” Source: Stefano Spaccapietra
- 27. 27Jarrar © 2013 Integration Methods: Semi-Automatic schemas integrated schema mapping rules TOOL DBA correspondences Opens to visual CASE tools, integration servers BUT knowledge acquisition can be painful Second method : semi-automatic integration “ tell me about the problem, I will try to fix it “
- 28. Jarrar © 2013 28 References and Acknowledgement • Carlo Batini: Course on Data Integration. BZU IT Summer School 2011. • Stefano Spaccapietra: Information Integration. Presentation at the IFIP Academy. Porto Alegre. 2005. • Chris Bizer: The Emerging Web of Linked Data. Presentation at SRI International, Artificial Intelligence Center. Menlo Park, USA. 2009. Thanks to Anton Deik for helping me preparing this lecture