On the Effectiveness of Evidence-based Terminological Decision Trees
0 likes174 views
The document presents a framework for evidence-based terminological decision trees (ETDTs) to predict class membership for individuals in description logics. ETDTs combine description logics, decision trees, and Dempster-Shafer theory. Experiments show ETDTs outperform previous approaches by assigning correct membership and limiting omission cases. While performance is similar to terminological decision trees when membership is definite, ETDTs induce better models. Future work includes further experiments, heuristics, combination rules and refinement operators.
![Introduction & Motivations
Motivations
Previous solutions and the current limits
We investigated the problem by resorting to a solution [Rizzo et
al.@IPMU14] which combines
Terminological Decision Tree (TDT): a DL-based Decision Tree for
concept learning and assertion prediction problems
the Dempster-ShaferTheory for dealing with missing values and making
the final assignment
Some limits:
inducing too conservative models that cannot predict a definite
membership
monotonicity of the functions used for class assignement
G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 4 / 18](https://image.slidesharecdn.com/rizzo-ismis15-151212173056/85/On-the-Effectiveness-of-Evidence-based-Terminological-Decision-Trees-4-320.jpg)
![DS & Description Logics
The Dempster-Shafer Theory (DS)
Frame of discernement Ω
a set of hypotheses for a domain, e.g. the membership values for an
individual given a concept Ω = {−1, +1}
Basic Belief Assignement (BBA) m : 2Ω → [0, 1]
the amount of belief exactly committed to A ⊆ Ω
Belief function: ∀A, B ∈ 2Ω Bel(A) = B⊆A m(B)
Plausibility function: ∀A, B ∈ 2Ω Pl(A) = B∩A=∅ m(B)
Confirmation function: Conf (A) = Bel(A) + Pl(A) − 1
G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 6 / 18](https://image.slidesharecdn.com/rizzo-ismis15-151212173056/85/On-the-Effectiveness-of-Evidence-based-Terminological-Decision-Trees-6-320.jpg)
![The framework
Learning Individual Classifiers
Given:
a target concept C
a training set Tr = Ps, Ns, Us for which the correct classification
tC (·) : Ind → {−1, 0, +1} is known:
Ps = {a ∈ Ind(A)|K |= C(a), i.e tC (a) = +1}
Ns = {b ∈ Ind(A)|K |= ¬C(b), i.e. tC (a) = −1}
Us = {c ∈ Ind(A)|K |= C(c) ∧ K |= ¬C(c), i.e. tC (c) = 0}
Find a classifier tC (·) : Ind → {−1, 0, +1} for C such that:
1
|Tr|
a∈Tr
1[hc(a) = tc(a)] > 1 −
G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 9 / 18](https://image.slidesharecdn.com/rizzo-ismis15-151212173056/85/On-the-Effectiveness-of-Evidence-based-Terminological-Decision-Trees-9-320.jpg)
On the Effectiveness of Evidence-based Terminological Decision Trees
- 1. On the Effectiveness of Evidence-based Terminological Decision Trees Giuseppe Rizzo, Claudia d’Amato, Nicola Fanizzi Dipartimento di Informatica Universit`a degli Studi di Bari ”Aldo Moro”, Bari, Italy ISMIS 2015 October 22nd, 2015 G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 1 / 18
- 2. Outline 1 Introduction & Motivations 2 DS & Description Logics 3 The framework 4 Experiments 5 Future Works G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 2 / 18
- 3. Introduction & Motivations Motivations AIM: predicting the membership of an individual w.r.t. a query concept in Description Logics typically based on automated reasoning techniques Inferences are affected by the incompleteness of the Semantic Web decided using models induced by Machine learning methods e.g methods borrowed by ILP which produces intensional definitions G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 3 / 18
- 4. Introduction & Motivations Motivations Previous solutions and the current limits We investigated the problem by resorting to a solution [Rizzo et al.@IPMU14] which combines Terminological Decision Tree (TDT): a DL-based Decision Tree for concept learning and assertion prediction problems the Dempster-ShaferTheory for dealing with missing values and making the final assignment Some limits: inducing too conservative models that cannot predict a definite membership monotonicity of the functions used for class assignement G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 4 / 18
- 5. Introduction & Motivations Motivations Underlying idea New procedure taking into account the monotonocity for deciding the class assignment in the prediction through the Evidence-based Terminological Decision Trees G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 5 / 18
- 6. DS & Description Logics The Dempster-Shafer Theory (DS) Frame of discernement Ω a set of hypotheses for a domain, e.g. the membership values for an individual given a concept Ω = {−1, +1} Basic Belief Assignement (BBA) m : 2Ω → [0, 1] the amount of belief exactly committed to A ⊆ Ω Belief function: ∀A, B ∈ 2Ω Bel(A) = B⊆A m(B) Plausibility function: ∀A, B ∈ 2Ω Pl(A) = B∩A=∅ m(B) Confirmation function: Conf (A) = Bel(A) + Pl(A) − 1 G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 6 / 18
- 7. DS & Description Logics The Dempster-Shafer Theory (DS) Combination rules: used for pooling evidences for the same frame of discernment coming from various sources of information Dubois-Prade’s rule ∀A, B, C ⊆ Ω m12(A) = B∪C=A m1(B)m2(C) G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 7 / 18
- 8. DS & Description Logics DL knowledge bases In DLs, a domain is modeled through concepts, i.e. classes of objects (e.g Man,Woman, Person) roles, i.e relationships between concepts (e.g. isFatherOf) individuals i.e. objects/resources (e.g GIUSEPPE) operators for obtaining more complex concept descriptions (e.g ¬Woman) A knowledge base K = (T , A) contains T , contains inclusion axioms D C (e.g Man Person) A, contain factual knowledge concerning individuals (Person(GIUSEPPE)) Reasoning services: instance-check test w.r.t. D, i.e., given an individual a, K |= D(a) ? G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 8 / 18
- 9. The framework Learning Individual Classifiers Given: a target concept C a training set Tr = Ps, Ns, Us for which the correct classification tC (·) : Ind → {−1, 0, +1} is known: Ps = {a ∈ Ind(A)|K |= C(a), i.e tC (a) = +1} Ns = {b ∈ Ind(A)|K |= ¬C(b), i.e. tC (a) = −1} Us = {c ∈ Ind(A)|K |= C(c) ∧ K |= ¬C(c), i.e. tC (c) = 0} Find a classifier tC (·) : Ind → {−1, 0, +1} for C such that: 1 |Tr| a∈Tr 1[hc(a) = tc(a)] > 1 − G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 9 / 18
- 10. The framework Evidence-based TDTs An ETDT is a binary tree where: each node contains a conjunctive concept description D and a BBA m; each departing edge is the result of instance-check test w.r.t. D a child node with the concept description D is obtained using a refinement operator G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 10 / 18
- 11. The framework An example of ETDT ∃hasPart. m= (∅: 0, {+1}:0.30,{-1}:0.36, {-1,+1}: 0.34) ∃hasPart.Worn m=(∅: 0.00, {+1}:0.50,{-1}:0.36, {-1,+1}: 0.14) ∃hasPart.(Worn ¬Replaceable) m=(∅: 0.00, {+1}:0.50,{-1}:0.36, {-1,+1}:0.00) SendBack m= (∅: 0.00, {+1}:1.00,{-1}:0.00, {-1,+1}:0.00) ¬SendBack m=(∅: 0.00, {+1}:0.00,{-1}:1.00, {-1,+1}:0.00) ¬SendBack m=(∅: 0.00, {+1}:0.00,{-1}:0.13, {-1,+1}:0.87) ¬SendBack m=(∅: 0.0, {+1}:0.00,{-1}:0.00, {-1,+1}: 1.0) Ω = {−1, +1} {+1} ↔ K |= D(a) ∀a ∈ Ind(A) {−1} ↔ K |= ¬D(b) ∀b ∈ Ind(A) {−1, +1} otherwise G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 11 / 18
- 12. The framework Learning ETDTs Divide-and-conquer algorithm for learning an ETDT Steps: 1 refinement of the concept description installed into the current node 2 A BBA for each selected description 3 The concept having the most definite membership (and its BBA) installed into the new node. Stop conditions: the node is pure w.r.t. the membership G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 12 / 18
- 13. The framework Predicting membership for unseen individuals Given an ETDT T and a new individual a A prediction returned by an ETDT is decided by following a path according to the instance check test result. For a concept description installed as node D if K |= D(a) the left branch is followed if K |= ¬D(a) the right branch is followed otherwise both branches are followed Various leaves can be reached and the corresponding BBAs are pooled according to the combination rule G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 13 / 18
- 14. The framework Predicting membership for unseen individuals After a pooled BBA m is obtained, Bel, Pl and Conf functions are derived Final membership assignement: hypothesis which maximizes confirmation function Bel and Pl function are monotonic : uncertain-memberhip is more probable Assign the uncertain membership when the values for the positive- and negative-membership are approximately equal decided according to a threshold on their difference G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 14 / 18
- 15. Experiments Experiments 20 query concepts randomly generated 10-fold cross validation Comparison w.r.t. TDTs, Celoe, Disjunctive EL Tree Learner(ELTL) Metrics: f-measure match: individuals for which the inductive model and a reasoner predict the same membership commission: cases of opposite predictions omission: individuals having a definite membership that cannot be predicted inductively; induction: predictions that are not logically derivable. G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 15 / 18
- 16. Experiments Experiments Outcomes Ontology Index TDT ETDT Celoe Disj ELTL BCO F1 67.54 ± 14.36 93.48 ± 09.46 100.0 ± 00.00 64.16 ± 02.15 M% 68.93 ± 15.87 94.53 ± 07.68 100.0 ± 00.00 47.27 ± 02.35 C% 06.14 ± 07.20 05.47 ± 07.68 00.00 ± 00.00 52.72 ± 02.35 O% 16.94 ± 09.74 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 I% 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 BioPax F1 96.67 ± 10.54 96.67 ± 10.54 38.02 ± 11.74 93.33 ± 14.05 M% 99.67 ± 01.02 99.67 ± 01.02 70.41 ± 10.97 99.37 ± 01.34 C% 00.32 ± 01.02 00.32 ± 01.02 29.59 ± 10.97 00.32 ± 01.02 O% 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 00.31 ± 00.99 I% 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 Financial F1 40.00 ± 51.64 100.0 ± 00.00 100.0 ± 00.00 100.0 ± 00.00 M% 67.06 ± 36.09 99.70 ± 00.48 99.70 ± 00.65 99.70 ± 00.68 C% 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 00.00 ± 00.00 O% 32.94 ± 36.09 00.00 ± 00.00 00.30 ± 00.68 00.30 ± 00.68 I% 00.00 ± 00.00 00.30 ± 00.50 00.00 ± 00.00 00.00 ± 00.00 G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 16 / 18
- 17. Experiments Discussion The new procedure tends to assign the correct membership the performance is good in terms of f-measure predict a definite membership: no omission cases and limited induction cases the performance of TDTs and ETDTs are similar only when all the individuals have a definite membership (e.g. due to a large number of disjointness axioms) better models than Celoe and Disjunctive ELTL very poor induced concept descriptions G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 17 / 18
- 18. Future Works Conclusions and Further Extensions An improvement of the previous inductive classification models for Semantic Web ontologies has been proposed Competitive w.r.t. the models obtained through state-of-the-art approaches Future works Further experiments Further heuristics for selecting the best concept Further combination rules Further refinement operators G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 18 / 18
- 19. End Thank you! Questions? G.Rizzo et al. (DIB - Univ. Aldo Moro) ISMIS 2015 October 22nd, 2015 18 / 18