Conceptual similarity: why, where and how
1 like2,850 views
The document discusses conceptual similarity and how, where, and why it is measured. It provides examples of measuring similarity between different types of objects, such as words, concepts, and hierarchies. It also summarizes various methods used to measure similarity, including precision and recall used in information retrieval, string matching measures to compare lexical terms, and measures to compare conceptual structures by examining hierarchies and semantic relationships between concepts.
![Comparing Hierarchies
*
car from the left hierarchy to the
node auto from the right hierarchy?
van from both hierarchies?
* [Dellschaft and Staab, 2006]](https://image.slidesharecdn.com/sfakakis-110331110716-phpapp01/85/Conceptual-similarity-why-where-and-how-7-320.jpg)
![Measuring Similarity,
Comparing two Ontologies
A simplified definition of a core ontology*:
The structure O := (C, root, C) is called a core ontology. C
is a set of concept identifiers and root is a designated root
concept for the partial order C on C. This partial order is
called concept hierarchy or taxonomy. The equation
c C : c C root holds for this concept hierarchy.
Levels of comparison
Lexical, how terms are used to convey meanings
Conceptual, which conceptual relations exist between terms
* [Dellschaft and Staab, 2006]](https://image.slidesharecdn.com/sfakakis-110331110716-phpapp01/85/Conceptual-similarity-why-where-and-how-14-320.jpg)
![Measuring Similarity,
Lexical Comparison Level String Matching
String Matching measure
(SM), given two lexical
entries L1, L2
Weights the number of the
required changes against
the shorter string
1 stands for perfect match,
0 for bad match
Average SM
Asymmetric, determines the
extend to which L1 (target)
is covered by L2 (source)
[Maedche and Staab, 2002]](https://image.slidesharecdn.com/sfakakis-110331110716-phpapp01/85/Conceptual-similarity-why-where-and-how-18-320.jpg)
![Measuring Similarity, Conceptual
Comparison Level local measures tp
OR OC
Local taxonomic precision examples using sc
OR) = {root, bike},
OC) = {root, bike, BMX}
tpsc C, OR) = |{root, bike}|/|{root, bike, BMX}|,
tpsc C, OR) = 2/3 = 0.67
[Maedche and Staab, 2002]](https://image.slidesharecdn.com/sfakakis-110331110716-phpapp01/85/Conceptual-similarity-why-where-and-how-25-320.jpg)
Conceptual similarity: why, where and how
- 1. Conceptual Similarity: Why, Where, How Michalis Sfakakis Laboratory on Digital Libraries & Electronic Publishing, Department of Archives and Library Sciences, Ionian University, Greece First Workshop on Digital Information Management, March 30-31, 2011 Corfu, Greece
- 2. Do we need similarity? Are the following objects similar? (Similarity, SIMILARITY) As character sequences, NO! How do they differ? As character sequences, but case insensitive, Yes! As English words, Yes! Same word! They have the same definition, written differently
- 3. Contents Introduction Disciplines How we measure similarity Focus on Ontology Learning evaluation
- 4. What about the similarity of the objects? (1, a) The first object is the number one and the second is the first letter of the English alphabet. Therefore, as the first is a number and the second is a letter, they are different! e.g. a chapter, or a paragraph number, they are both representing the first object of the list, the first chapter, paragraph, etc. Therefore, they could be considered as being similar!
- 5. Results for an Information Need How similar are the Results? Which one to select?
- 6. Comparing Concepts objects? (Disease, Illness) As English words, or as character sequences they are not similar! How do they differ? As synonymous terms in a Thesaurus, they are both representing the same concept. (related with the equivalency relationship)
- 7. Comparing Hierarchies * car from the left hierarchy to the node auto from the right hierarchy? van from both hierarchies? * [Dellschaft and Staab, 2006]
- 8. Similarity is a context dependent concept Merriam- defines similarity as*: A quality that makes one person or thing like another Therefore, the context and the characteristics in common are required in order to specify and measure similarity * http://www.learnersdictionary.com/search/similarity
- 9. Where the concept of similarity is encountered Machine learning Ontology Learning Schema & Ontology Matching and Mapping Clustering IR pattern recognition algorithm Vital part of the Semantic Web development
- 10. Precision & Recall in IR, measuring similarity between answers Let C be the result set for a query (the retrieved documents, i.e. the Computed set) Also, we need to know the correct results for the query (all the relevant documents, the Reference set) Precision: is the fraction of retrieved documents that are relevant to the search Recall: is the fraction of the documents that are relevant to the query that are successfully retrieved Wikipedia: http://en.wikipedia.org/wiki/Precision_and_recall
- 11. measure similarity Precision & Recall are two widely used metrics for evaluating the correctness of a pattern recognition algorithm Recall and Precision depend on the outcome (oval) of a pattern recognition algorithm and its relation to all relevant patterns (left) and the non-relevant patterns (right). The more correct results (green), the better. Precision: horizontal arrow. Recall: diagonal arrow. Wikipedia: http://en.wikipedia.org/wiki/Precision_and_recall
- 12. Precision & Recall, once more D Precision P = |R C|/|R| False Negative False Positive Recall R = |R C|/|C| R R C C TP = R C TN = D (R C) True Positive FN = R C FP = C R True Negative
- 13. Overall evaluation, combining Precision & Recall Given Precision & Recall, F-measure could combines them for an overall evaluation Balanced F-measure (P & R are evenly weighted) F1 = 2*(P*R)/(P+R) Weighted F-measure Fb = (1+b2)*(P*R)/(b2*P+R), b non-zero F1 (b=2) weights recall twice as much as precision F0.5 (b=0.5) weights precision twice as much as recall
- 14. Measuring Similarity, Comparing two Ontologies A simplified definition of a core ontology*: The structure O := (C, root, C) is called a core ontology. C is a set of concept identifiers and root is a designated root concept for the partial order C on C. This partial order is called concept hierarchy or taxonomy. The equation c C : c C root holds for this concept hierarchy. Levels of comparison Lexical, how terms are used to convey meanings Conceptual, which conceptual relations exist between terms * [Dellschaft and Staab, 2006]
- 15. Gold Standard based Evaluation of Ontology Learning OR OC Given a pre-defined ontology The so-called Gold Standard or Reference Compare the Learned (Computed) Ontology with the Gold Standard
- 16. Measuring Similarity - Lexical Comparison Level LP, LR OR OC Lexical Precision & Lexical Recall LP(OC, OR) = |CC CR|/|CC| LR(OC, OR) = |CC CR|/|CR| The lexical precision and recall reflect how good the learned lexical terms CC cover the target domain CR For the above example LP=4/6=0.67, LR=4/5=0.8
- 17. Measuring Similarity, Lexical Comparison Level - aSM Average String Matching, using edit distance Levenshtein distance, the most common definition for edit distance, measures the minimum number of token insertions, deletions and substitutions required to transform one string into an other For example*, the Levenshtein distance between "kitten" and "sitting" is 3 (there is no way to do it with fewer than three edits) kitten sitten (substitution of 's' for 'k') sitten sittin (substitution of 'i' for 'e') sittin sitting (insertion of 'g' at the end). * Wikipedia: http://en.wikipedia.org/wiki/Levenshtein_distance
- 18. Measuring Similarity, Lexical Comparison Level String Matching String Matching measure (SM), given two lexical entries L1, L2 Weights the number of the required changes against the shorter string 1 stands for perfect match, 0 for bad match Average SM Asymmetric, determines the extend to which L1 (target) is covered by L2 (source) [Maedche and Staab, 2002]
- 19. Measuring Similarity, Lexical Comparison Level - RelHit RelHit actually express Lexical Precision RelHit Compared to average String Matching Average SM reduces the influences of string pseudo-differences (e.g. singular vs. plurals) Average SM may introduce some kind of noise,
- 20. Measuring Similarity, Conceptual Comparison Level Conceptual level compares semantic structure of ontologies Conceptual structures are constituted by Hierarchies, or by Relations How to compare two hierarchies? How do the positions of concepts influence similarity of Hierarchies? What measures to use?
- 21. Measuring Similarity, Conceptual Comparison Level OR OC Local measures compare the positions of two concepts based on characteristics extracts from the concept hierarchies they belong to Some characteristic extracts Semantic Cotopy (sc) sc(c, O) = {ci|ci C (ci c c ci)} Common Semantic Cotopy (csc) csc(c, O1, O2) = {ci|ci C1 C2 (ci c c ci)}
- 22. Measuring Similarity, Conceptual Comparison Level sc OR OC Semantic Cotopy sc(c, O) = {ci|ci C (ci c c ci)} Semantic Cotopy examples OR) = {root, bike, car, van, coup} OC) = {root, bike, auto, BMX, van, coup } OR) = {root, bike} OC) = {root , bike, BMX} OR) = {root , car, van, coup } OC) = {root, auto, van, coup }
- 23. Measuring Similarity, Conceptual Comparison Level csc OR OC Common Semantic Cotopy csc(c, O1, O2) = {ci|ci C1 C2 (ci c c ci)} Common Semantic Cotopy examples C1 C2 = {root, bike, van, coup } OR, OC) = {bike, van, coup } OC, OR) = {bike, van, coup } OR, OC OC, OR) = {root} OR, OC) = {root , van, coup OC, OR) = Oc, OR) = {root, van, coup OC, OR) =
- 24. Measuring Similarity, Conceptual Comparison Level local measures tp, tr OR OC Local taxonomic precision using characteristic extracts tpce(c1, c2, OC, OR) = |ce(c1, OC) ce(c1, OR) |/|ce(c1, OC)| Local taxonomic recall using characteristic extracts trce(c1, c2, OC, OR) = |ce(c1, OC) ce(c1, OR) |/|ce(c1, OR)|
- 25. Measuring Similarity, Conceptual Comparison Level local measures tp OR OC Local taxonomic precision examples using sc OR) = {root, bike}, OC) = {root, bike, BMX} tpsc C, OR) = |{root, bike}|/|{root, bike, BMX}|, tpsc C, OR) = 2/3 = 0.67 [Maedche and Staab, 2002]
- 26. Measuring Similarity, Conceptual Comparison Level local measures tp OR OC Local taxonomic precision examples using sc OR) = {root , car, van, coup }, OC) = {root , auto, van, coup } tpsc car C, OR) = |{ } |/| |, tpsc car C, OR) = 3/4 = 0.75
- 27. Measuring Similarity, Conceptual Comparison Level comparing Hierarchies OR OC Global Taxonomic Precision (TP)
- 28. Measuring Similarity, Conceptual Comparison Level Overall evaluation F-measure, but now using Global Taxonomic Precision (TP) and Global Taxonomic Recall (TR) Balanced Taxonomic F-measure (TP & TR are evenly weighted) TF1 = 2*(TP*TR)/(TP+TR) Weighted TF-measure TFb = (1+b2)*(TP*TR)/(b2*TP+TR), b non-zero TF1 (b=2) weights recall twice as much as precision TF0.5 (b=0.5) weights precision twice as much as recall
- 29. Measuring Similarity, Conceptual Comparison Level Taxonomic Overlap taxonomic overlap (TO)
- 30. References & Further Reading Dellschaft, Klaas and Staab, Steffen ( 2006) : On How to Perform a Gold Standard Based Evaluation of Ontology Learning. In: I. Cruz et al. (Eds.) ISWC 2006. LNCS 4273, pp. 228 241. Springer, Heidelberg Maedche, A., Staab, S. (2002): Measuring similarity between ontologies. In: Proceedings of the European Conference on Knowledge Acquisition and Management (EKAW-2002). Siguenza, Spain Staab, S. and Hotho, A.: Semantic Web and Machine Learning Tutorial (available at http://www.uni- koblenz.de/~staab/Research/Events/ICML05tutorial/icml05tuto rial.pdf) Bellahsene, Z., Bonifati, A., Rahm, E. (Eds.) (2011): Schema Matching and Mapping, Heidelberg, Springer- Verlag, ISBN 978-3-642-16517-7.
- 31. End of tutorial! Thanks for your attention! Michalis Sfakakis