History of text summarization
- 1. H I S T O R Y O F T E X T S U M M A R I Z AT I O N FA N G YA TA N & W E N L I
- 2. PRESENTATION FLOW • Introduction of Text Summarization History ~Hans Peter Luhn in the late 1950s, titled “The automatic creation of literature abstracts” • Fundamental Question of Text Summarization? ~ Select the essential content , Express the selected content • Two Main Types of Text summarization Methods ~ extractive and abstractive • Interesting Text Summarization Example: ~ A Dream in the Red Chamber • Q and A
- 3. INTRODUCTION • What is Text Summarization? Automatic summarization is the process of shortening a text document with software, to create a summary with the major points of the original document. –Wikipedia • Why it is Important? It can quickly extract accurate content and help reader understand large volume of information. (reviews, news headline, biography) History: Automation Summarization 1950, Text Analysis conferences 2000, Today& Future: Automation with AI Algorithm: CNN, RNN, HAN
- 4. S ELECT S E NTENCE O R E X PRESS T H E CO N TENT? Evaluation: • Summarization Goal Syntactically accurate Semantically coherent Logically organized Without redundancy Measure System ROUGE: Recall- Oriented Understudy for Gist Evaluation Pyramid Automatic Evaluation with Machine Learning (supervised or unsuperised) Challenges : How to select the essential content of a document ? How to express the selected content in a condensed manner?
- 5. FUNDAMENTAL TYPES TEXT SUMMARIZATION METHODS EXTRACTIVE SUMMARIZATION • important sentences are selected from the input text to form a summary • Input document → sentences similarity → weight sentences → select sentences with higher rank • Application: Easier to use • Challenges: Identify the right sentence • Weakness: Low readability ABSTRACTIVE SUMMARIZATION • Model generate text summarization from its main ideas like human. • Input document → understand context → semantics → create own summary • Application: English & Arabics • Approaches: Structured, Semantic • Challenges: Need complex capabilities like generalization, paraphrasing and incorporating real-world knowledge. Based on the output type, we can classify text summarization into Tools: Python, NLTK
- 6. A DREAM OF THE RED CHAMBER: FIRST 80 VS LAST 40 CHAPTERS? WORD FREQUENCY FIRST 80 VS L AST 40: • T H R O U G H N L P I R PAC K AG E , T H E R E S E A R C H E R S A P P L I E D K L D I V E R G E N C E A N D P C A D I M E N S I O N ( 1 3 8 ) R E D U C T I O N D E M O N S T R AT E D A S I G N I F I C A N T D I F F E R E N C E B E T W E E N F I R S T 8 0 A N D L A S T 4 0 C H A P T E R S
- 7. R E F E R E N C E : H T T P S : / / W W W . A N A L Y T I C S V I D H Y A . C O M / B L O G / 2 0 1 8 / 1 1 / I N T R O D U C T I O N - T E X T - S U M M A R I Z A T I O N - T E X T R A N K - P Y T H O N / A U T O M A T I C T E X T S U M M A R I Z A T I O N P A S T P R E S E N T A N D F U T U R E H T T P S : / / E N . W I K I P E D I A . O R G / W I K I / A U T O M A T I C _ S U M M A R I Z A T I O N H T T P S : / / W W W . Z H I H U . C O M / Q U E S T I O N / 1 9 7 6 8 8 9 8 H T T P S : / / F R E E W E C H A T . C O M / A / M Z I 5 N D Q 5 M J I 5 O Q = = / 2 2 4 7 4 8 4 5 9 4 / 3 H T T P S : / / W W W . I D A T A L O O P . C O M / 1 4 2 6 0 . H T M L T H A N K Y O U !