Query Expansion with Locally-Trained Word Embeddings (ACL 2016)
4 likes1,127 views
This document discusses using locally-trained word embeddings for query expansion to improve search results. It trains word embeddings on either the full corpus (global) or just the topically-relevant documents (local) and finds the local approach works better. It experiments on three datasets and finds local embeddings consistently outperform global embeddings and no expansion for search result ranking based on NDCG.
![• local term clustering [Lesk 1968, Attar and Fraenkel
1977]
• local latent semantic analysis [Hull 1995, Hull, 1994;
Schutze et al., 1995; Singhal et al., 1997]
• local document clustering [Tombros and van
Rijsbergen, 2001; Tombros et al., 2002; Willett, 1985]
• one sense per discourse [Gale et al., 1992]](https://image.slidesharecdn.com/acl2016-fdiaz-160808141157/85/Query-Expansion-with-Locally-Trained-Word-Embeddings-ACL-2016-8-320.jpg)
![q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …]
query = gas tax](https://image.slidesharecdn.com/acl2016-fdiaz-160808141157/85/Query-Expansion-with-Locally-Trained-Word-Embeddings-ACL-2016-10-320.jpg)
![q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …]
query = gas tax
d = [gas:0.0 tax:0.0 petroleum:0.7 tariff:0.5 …]](https://image.slidesharecdn.com/acl2016-fdiaz-160808141157/85/Query-Expansion-with-Locally-Trained-Word-Embeddings-ACL-2016-11-320.jpg)
![q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …]
query = gas tax
…
gas petroleum:0.9 indigestion:0.6 …
tax tariff:0.7 strain:0.4 …
…[ ]W=](https://image.slidesharecdn.com/acl2016-fdiaz-160808141157/85/Query-Expansion-with-Locally-Trained-Word-Embeddings-ACL-2016-12-320.jpg)
![q = [gas:1.0 tax:1.0 petroleum:0.8 tariff:0.6 …]
query = gas tax
d = [gas:0.0 tax:0.0 petroleum:0.7 tariff:0.5 …]](https://image.slidesharecdn.com/acl2016-fdiaz-160808141157/85/Query-Expansion-with-Locally-Trained-Word-Embeddings-ACL-2016-13-320.jpg)
Query Expansion with Locally-Trained Word Embeddings (ACL 2016)
- 1. Query Expansion with Locally- Trained Word Embeddings Fernando Bhaskar Mitra Nick Craswell Microsoft
- 2. p(d) d
- 4. cut global local* cutting tax squeeze deficit reduce vote slash budget reduction reduction spend house lower bill halve plan soften spend freeze billion global: trained using full corpus local: trained using topically- *gas
- 5. global local t-SNE projection: top words by ˜p(d|q) (blue: query; red: top words by p(d|q))
- 8. • local term clustering [Lesk 1968, Attar and Fraenkel 1977] • local latent semantic analysis [Hull 1995, Hull, 1994; Schutze et al., 1995; Singhal et al., 1997] • local document clustering [Tombros and van Rijsbergen, 2001; Tombros et al., 2002; Willett, 1985] • one sense per discourse [Gale et al., 1992]
- 10. q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …] query = gas tax
- 11. q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …] query = gas tax d = [gas:0.0 tax:0.0 petroleum:0.7 tariff:0.5 …]
- 12. q = [gas:1.0 tax:1.0 petroleum:0.0 tariff:0.0 …] query = gas tax … gas petroleum:0.9 indigestion:0.6 … tax tariff:0.7 strain:0.4 … …[ ]W=
- 13. q = [gas:1.0 tax:1.0 petroleum:0.8 tariff:0.6 …] query = gas tax d = [gas:0.0 tax:0.0 petroleum:0.7 tariff:0.5 …]
- 14. W = UUT U m ⇥ k embedding matrix
- 15. p(d) d q p(d|q)
- 16. p(d) d q ˜p(d|q)
- 18. U = 8 >>>< >>>: uniform p(d) on the target corpus uniform p(d) on an external corpus p(d|q) on the target corpus p(d|q) on an external corpus
- 19. docs words queries trec12 469,949 438,338 150 robust 528,155 665,128 250 web 50,220,423 90,411,624 200
- 20. global local target target wikipedia+gigaword* gigaword† google news* wikipedia† *publicly available embedding; †publicly available external corpus target corpus query results external corpus query results target corpus query results target corpus query results external corpus query results
- 21. trec12 robust web local vs global NDCG@10 0.0 0.1 0.2 0.3 0.4 0.5 expansion none global local
- 22. trec12 robust web local embedding NDCG@10 0.0 0.1 0.2 0.3 0.4 0.5 corpus target gigaword wikipedia
- 23. • local embedding provides a stronger representation than global embedding • potential impact for other topic-specific natural language processing tasks • future work • effectiveness improvements • efficiency improvements