AttSum: Joint Learning of Focusing and Summarization with Neural Attention
0 likes307 views
The document presents a novel approach to extractive query-focused summarization using a joint neural network model called 'attsum', which incorporates an attention mechanism to simulate human reading behavior. It ranks sentences based on query relevance and saliency, employing a training process focused on maximizing overview quality. The evaluation of the model is conducted using the ROUGE metric on datasets from the DUC 2005-2007 competitions.
AttSum: Joint Learning of Focusing and Summarization with Neural Attention
- 1. AttSum: Joint Learning of Focusing and Summarization with Neural Attention Ziqiang Cao, Wenjie Li, Sujian Li, Furu Wei and Yanran Li Coling 2016 発表者：小平 知範 1
- 2. Abstract • Task: Extractive query-focused summarization - query relevance ranking and sentence saliency ranking • Main contributions: - They apply the attention mechanism that tries to simulate human attentive reading behavior for query-focused summarization. - They propose a joint neural network model to learn query relevance ranking and sentence saliency ranking simultaneously 2
- 3. Query-Focused Sentence Ranking CNN Layer Pooling Layer Ranking Layer 3
- 4. 1. CNN Layer 3. Query-Focused Sentence Ranking 4 v(wi: wi+j) = concatenation Convolution Layer Max-over-time pooling f (●) = non-linear function Wt h ∈ Rl x hk h = window size k = embeding size ct h ∈ Rl
- 5. 2. Pooling Layer • Query relevance: • The document Embedding: 3. Query-Focused Sentence Ranking 5 M ∈ Rl x l
- 6. 3. Ranking Layer • rank a sentence according to cosine similarity • Cosine Similarity: 3. Query-Focused Sentence Ranking 6
- 7. ex. Training Process • Cost Function: • s+: High ROUGE score, s-: rest • Ω is margin threthold 3. Query-Focused Sentence Ranking 7
- 8. Sentence Selection • 1. discard sentences less than 8 words • 2. sort descending order • 3. They iteratively dequeue the top-ranked sentence, and append it to the current summary if it is non-redundant. • non-redundant: new bi-gram ratio > .5 8
- 9. Dataset • DUC 2005 ~ 2007, query-focused multi-document summarization task. • Preprocessing: StanfordCoreNLP (ssplit, tokenize) • Summary: the length limit of 250 words • Validation: 3-fold cross-validation 4. Experiments 9
- 10. Model Setting • Word Embedding: (50 dimention, trained on News corpus) - don’t update word embeddings in the training process • word window size h = 2 • Convolution output l = 50 • margin Ω = 0.5 10 4. Experiments
- 11. Evaluation Metrics • ROUGE-2 11 4. Experiments
- 12. Baselines • LEAD • QUERY_SIM • MultiMR (Wan and iao, 2009) • SVR (Ouyang et al., 2011) • DocEmb (Kobayashi et al., 2015) • ISOLATION: AttSum w/o attention mechanism 12 4. Experiments
- 14. Conclusion • Propose a novel query-focuesed summarization system called AttSum, which jointly handles saliency ranking and relevance ranking. 14