Sequential Query Expansion using Concept Graph
0 likes553 views
The document describes a proposed method for sequential query expansion using a concept graph. The method involves two stages: 1) initially sorting concepts in each layer of the concept graph by predicted quality score, and 2) sequentially selecting concepts using a decision criterion that either continues with the current concept layer, moves to the next layer, or stops expansion. The goal is to optimize retrieval performance while minimizing the number of evaluated concepts. The method is intended to address challenges in efficiently exploring a large concept graph during query expansion.
Sequential Query Expansion using Concept Graph
- 1. 1/20 Introduction Method Experiments Conclusions Sequential Query Expansion using Concept Graph Saeid Balaneshin-kordan Alexander Kotov Wayne State University October 25, 2016 Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 2. 2/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 3. 3/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 4. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 5. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 6. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 7. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 8. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 9. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 10. 5/20 Introduction Method Experiments Conclusions Challenges § The number of candidate concepts to evaluate increases exponentially with the number of layers to consider. § Only a small fraction of hundreds or potentially thousands of concepts that can improve retrieval results, while others need to be discarded to avoid noise and concept drift. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 11. 6/20 Introduction Method Experiments Conclusions Optimization Problem § Objective Function: total number of evaluated concepts Constraint: precision of retrieval results min ˜Cut k " kÿ i=1 Ni * s.t. E(˜RΛ; T) ą θQ , § Approximate Solution: Decision Criterion Select concept C(i,j) & If ˜Qr(C(i,j)) ě βUcontinue with the same concept layer Discard concept C(i,j) & If βL ď ˜Qr(C(i,j)) ă βUcontinue with the same concept layer Discard concept C(i,j) & If ˜Qr(C(i,j)) ă βLmove to the next concept layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 12. 7/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 13. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 14. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 15. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 16. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action Select Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 17. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action Select Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 18. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 19. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 20. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 21. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 22. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 23. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 24. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 25. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 26. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 27. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 28. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 29. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 30. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscardSTOP Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 31. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscardSTOP Number of Evaluated Concepts: 11 (26% less) Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 32. 9/20 Introduction Method Experiments Conclusions Summary of the proposed method and the baselines Method Optimization Problem Criteria in the Approximate Solution Objective Constraint Selecting Rejecting Stopping Method A mint řk i=0 Liu E( ˜Rk Λ; T) ą θ Qb(c) ą βQ Qb(c) ă βQ i ą k Method B maxtE( ˜Rk Λ; T)u řk i=0 Li ă θ Ii(c) ă βI Ii(c) ą βI i ą k Method C mint řk i=0 Liu E( ˜Rk Λ; T) ą θ Qb(c) ą βQ Qb(c) ă βQ i ą k Method D maxtE( ˜Rk Λ; T)u řk i=0 Li ă θ I(c) ă βI I(c) ą βI i ą k Proposed mint řk i=0 Niu E( ˜Rk Λ; T) ą θ Qr(c) ą βU Qr(c) ă βL Li = 0 § Qb(C(i,j)): Quality measure computed as a linear weighted combination of the feature functions. § I(c): Index of a concept in the sorted set of concepts § Li: Number of selected concepts from the i-th concept layer. § the set of features used to calculate the quality measure Qb(c) for the baselines is the same as the set of features used to calculate Qs(c) in for our proposed method. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 33. 10/20 Introduction Method Experiments Conclusions Baselines (With Fixed Number of Layers) Selection Region Selection Region Rejection Region Selection Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Selection Region Rejection Region Selection Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Rejection Region (1,1) (3,1) (2,1) (1,2) (2,2) (3,2) (2,3) (2,4) (2,5) (1,3) (3,3) (4,1) (4,2) (2,6) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Rejection Region (1,1) (3,1) (2,1) (1,2) (2,2) (3,2) (2,3) (2,4) (2,5) (1,3) (3,3) (4,1) (4,2) (2,6) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) SingleThreshold onEachLayer SingleThreshold onAllLayers A B C D Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 34. 11/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 35. 12/20 Introduction Method Experiments Conclusions Inexpensive Features § Retrieval score of the highest ranked document containing C(i,j) § Avg retrieval score of all documents containing C(i,j) § Variance of retrieval score of all documents containing C(i,j) § Avg retrieval scores of the top documents containing C(i,j) § Number of occurrences of C(i,j) in the top documents § Number of top documents containing C(i,j) § Node degree of C(i,j) in the concept graph § Avg number of paths between C(i,j) and query concepts § Max number of paths between C(i,j) and query concepts Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 36. 13/20 Introduction Method Experiments Conclusions Expensive Features § Avg co-occurrence of C(i,j) with query concepts § Max co-occurrence of C(i,j) with query concepts § Avg co-occurrence of C(i,j) with query concepts in top-docs § Max co-occurrence of C(i,j) with query concepts in top-docs § Avg co-occurrence of C(i,j) with at least a pair of query concepts in top-docs § Max co-occurrence of C(i,j) with at least a pair of query concepts in top-docs § Avg co-occurrence of C(i,j) with all previously selected concepts in top-docs § Max co-occurrence of C(i,j) with all previously selected concepts in top-docs § Avg co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 § Max co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 § Avg co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Max co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Avg mutual information of C(i,j) with at least a pair of query concepts in top-docs § Max mutual information of C(i,j) with at least a pair of query concepts in top-docs § Avg mutual information of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Max mutual information of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 37. 14/20 Introduction Method Experiments Conclusions Feature Ablation on ROBUST04 collection hgstDocScore maxTCooccur maxCooccurL* nodeDegree maxTCooccurL* maxNumLinks maxTCooccur* maxCooccur* varDocScore avgTCooccurL* avgCooccurL* maxTMiL* avgCooccur* avgTMiL* avgNumLinks maxTCooccurP* avgTCooccur* maxTMiP* avgDocScore avgTDocScore avgTCooccurP* avgTCooccur avgTMiP* docFreqTpDoc termFreqTpDoc none 0.15 0.20 0.25 0.30 MAP Feature The feature that results in the highest reduction of MAP after being removed from the feature set: § The features that are utilized in both stages of the proposed method § The features that are dependent on the collection Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 38. 15/20 Introduction Method Experiments Conclusions The impact of the upper and lower thresholds on MAP (i.e., βU and βL) at the concept layer i = 2 (e) TREC7-8 (f) ROBUST04 (g) GOV § Upper threshold < the optimum value: more non-useful concepts are added to the candidate list of expansion concepts. § Upper threshold > the optimum value: some useful concepts may not be selected as expansion concepts. § Lower threshold < the optimum value: the selection process may terminate earlier and a number of useful concepts may not be examined at all. § Lower threshold > the optimum value: the proposed method will evaluate more concepts in total, which is against its main objective. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 39. 15/20 Introduction Method Experiments Conclusions The impact of the upper and lower thresholds on MAP (i.e., βU and βL) at the concept layer i = 2 (h) TREC7-8 (i) ROBUST04 (j) GOV § Overall, although the upper and lower thresholds are dependent on each other, the upper threshold has the main effect on the accuracy of selected concepts, while the lower threshold has the main effect on the number of examined concepts. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 40. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th TREC7-8 Method D-HAL 0.2220 0.2239 0.2155 0.2120 Method D-CNet 0.2205 0.2245 0.2214 0.2183 Method C-HAL 0.2152 0.2227 0.2185 0.2133 Method C-CNet 0.2182 0.2265 0.2225 0.2218 Method B-HAL 0.2207 0.2171 0.2266 0.2236 Method B-CNet 0.2188 0.2294 0.2255 0.2294 Method A-HAL 0.2172 0.2251 0.2290 0.2282 Method A-CNet 0.2183 0.2290 0.2329 0.2335 Proposed-HAL 0.2249 0.2348 0.2418 0.2457 Proposed-CNet 0.2222 0.2377 0.2449 0.2484 SDM 0.2124 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 41. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th ROBUST04 Method D-HAL 0.2660 0.2644 0.2569 0.2554 Method D-CNet 0.2640 0.2651 0.2568 0.2555 Method C-HAL 0.2675 0.2655 0.2608 0.2516 Method C-CNet 0.2637 0.2628 0.2683 0.2695 Method B-HAL 0.2684 0.2718 0.2598 0.2535 Method B-CNet 0.2616 0.2710 0.2665 0.2675 Method A-HAL 0.2614 0.2758 0.2757 0.2764 Method A-CNet 0.2689 0.2732 0.2851 0.2793 Proposed-HAL 0.2721 0.2786 0.2865 0.2898 Proposed-CNet 0.2748 0.2814 0.2889 0.2963 SDM 0.2359 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 42. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th GOV Method D-HAL 0.2337 0.2428 0.2355 0.2319 Method D-CNet 0.2348 0.2396 0.2355 0.2382 Method C-HAL 0.2404 0.2406 0.2459 0.2322 Method C-CNet 0.2416 0.2451 0.2378 0.2379 Method B-HAL 0.2359 0.2466 0.2418 0.2397 Method B-CNet 0.2420 0.2452 0.2484 0.2421 Method A-HAL 0.2434 0.2442 0.2491 0.2420 Method A-CNet 0.2365 0.2455 0.2524 0.2422 Proposed-HAL 0.2455 0.2429 0.2570 0.2578 Proposed-CNet 0.2449 0.2514 0.2575 0.2591 SDM 0.2184 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 43. 17/20 Introduction Method Experiments Conclusions Retrieval Performance Without PRF Concepts Collection Evaluation QL SDM Method A Method A Proposed Proposed Metric HAL CNet HAL CNet TREC7-8 MAP 0.1982 0.2124 0.2282˚: 0.2335˚: 0.2457˚: 0.2484˚: (7.44%) (9.93%) (15.68%/7.67%) (16.95%/6.38%) P@20 0.3540 0.3765 0.3762 0.3783 0.3785˚ 0.3796˚ (-0.08%) (0.48%) (0.53%/0.61%) (0.82%/0.34%) ROBUST04 MAP 0.2359 0.2510 0.2764˚: 0.2851˚: 0.2898˚: 0.2963˚: (10.12%) (13.59%) (15.46%/4.85%) (18.05%/3.93%) P@20 0.3339 0.3667 0.3679 0.3773˚: 0.3802˚: 0.3795˚: (0.33%) (2.89%) (3.68%/3.34%) 3.49%/0.58% GOV MAP 0.2184 0.2333 0.2491˚ 0.2524˚: 0.2578˚: 0.2591˚: (6.77%) (8.19%) (10.5%/3.49%) 11.06%/2.65% P@20 0.0389 0.0451 0.0476 0.0493˚ 0.0558˚: 0.0552˚: (5.54%) (9.31%) (23.73%/17.23%) 22.39%/11.97% § Method A provides a significant improvement over SDM in the 5 cases. § Although the parameters are estimated with the goal of maximizing MAP, the proposed method demonstrates significant improvement over the baselines (QE and SDM) also in terms of P@20. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 44. 17/20 Introduction Method Experiments Conclusions Retrieval Performance With PRF Concepts Collection Evaluation RM LCE Method A* Method A* Proposed* Proposed* Metric HAL CNet HAL CNet TREC7-8 MAP 0.2151 0.2423 0.2503˚ 0.2558˚: 0.2642˚: 0.2672˚: (3.3%) (5.57%) (9.04%/5.55%) 10.28%/4.46% P@20 0.3641 0.3836 0.3883 0.3927˚ 0.3934˚: 0.4035˚: (1.23%) (2.37%) (2.55%/1.31%) 5.19%/2.75% ROBUST04 MAP 0.2683 0.2826 0.2935˚ 0.2979˚ 0.3034˚: 0.3053˚: (3.86%) (5.41%) (7.36%/3.37%) 8.03%/2.48% P@20 0.3561 0.3785 0.3826˚ 0.3834˚ 0.3893˚: 0.3965˚: (1.08%) (1.29%) (2.85%/1.75%) 4.76%/3.42% GOV MAP 0.2403 0.2678 0.2693 0.2730˚ 0.2793˚: 0.2811˚: (0.56%) (1.94%) (4.29%/3.71%) 4.97%/2.97% P@20 0.0483 0.0566 0.0583 0.0617˚ 0.0706˚ 0.0720˚: (3.00%) (9.01%) (24.73%/21.1%) 27.21%/16.69% § The proposed method has significant improvements over the baselines QL and SDM whether the concept graph is generated by HAL or ConceptNet. § Method A provides a significant improvement over LCE only in one of the cases when it uses PRF concepts. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 45. 18/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 46. 19/20 Introduction Method Experiments Conclusions Conclusions § The main contribution of this work: § A two-stage method for sequential selection of effective concepts for query expansion from the concept graph. § The optimization problem of the proposed method: § Objective: having least possible number of candidate concepts § Constraint: achieve a given precision of retrieval results. § Stages of the proposed method: § First Stage: the candidate concepts are sorted using a number of computationally inexpensive features. § Second Stage: This sorting is utilized in the second stage to sequentially select expansion concepts by using computationally expensive features. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 47. 19/20 Introduction Method Experiments Conclusions Conclusions § Experimental evaluation indicates that: § The proposed method outperforms state-of-the-art baselines, which instead of minimizing the number of evaluated concepts, aim to minimize the number of selected concepts or maximize a concept quality measure. § The proposed method and the baselines produce more accurate results using ConceptNet-based than collection-based concept graph. § Future Work: § We believe that applying the proposed method to the case of entity-based queries and knowledge graphs is an interesting future direction for extending this work. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
- 48. Many Thanks to the ACM SIGIR Student Travel Grant! 20/20