Deep Learning for Information Extraction in Natural Language Text
- 1. Deep Learning for Information Extraction in Natural Language Text Pankaj Gupta CT RDA BAM MIC-DE Young Research Forum 2017 | Siemens AG, Germany Siemens Corporate TechnologyRestricted © Siemens AG 2016
- 2. Restricted © Siemens AG 2016 31.05.2016Page 2 Corporate Technology About Me @CT RDA BAM MIC-DE !! Nov, 2015 Present ➢ LinkedIn: https://www.linkedin.com/in/pankaj-gupta-6b95bb17/ ➢ Google Scholar: https://scholar.google.com/citations?user=_YjIJF0AAAAJ&hl=en Research Focus: Machine Learning (Deep Learning) techniques to solve Natural Language Processing (NLP) tasks • Bachelor of Technology in Computer Science, Amity University, India • Bachelor Internship: Queen’s University, Belfast Northern Ireland UK • Publications: 2 • Senior Software Engineer at Wipro and Aricent Technologies, IndiaJun, 2010- Sept, 2013 • Master of Science in Informatics, Technical University of Munich (TUM), Germany • Master’s Thesis (Siemens + LMU): Deep Learning Methods for the Extraction of Relations in Natural Language Text • Publications: Deep Learning/NLP focused: 2 and Machine Learning based: 2 Oct, 2013 Nov, 2015 • PhD at CT-RDA-BAM-MIC-DE Siemens AG and at LMU (CIS) Munich Germany • Advisor(s): Dr. Bernt Andrassy (Siemens AG) and Prof. Dr. Hinrich Schütze (CIS LMU) • PhD Thesis: Deep Learning Methods for Information Extraction in Natural Language Text • Publications: Published: 1, Review: 3, Filed 3 patents Oct, 2006- April, 2010
- 3. Restricted © Siemens AG 2016 31.05.2016Page 3 Corporate Technology Information Extraction(IE) in Natural Language Text • Entity Extraction • Relation Extraction • Structure the unstructured text • Knowledge Graph Construction • In web search, retrieval, Q&A, etc. Information Extraction Entity Extraction: Detect entities such as person, organization, location, product, technology, sensor, etc. Relation Extraction: Detect relation between the given entities or nominals End-to-End Knowledge Base Population Text Documents Knowledge GraphIE Engine SensorSensor Competitor-of Sensor
- 4. Restricted © Siemens AG 2016 31.05.2016Page 4 Corporate Technology Supervised Deep Learning Techniques in Information Extraction • Natural language is sparse and noisy • Better Representation Learning • Build state-of-the-art entity and relation extraction systems with Neural Networks to extract triples (entity1, entity2, relation) Challenges and Motivation Our Pipelined Deep Learning System for Entity and Relation Extraction Motivation to build the state-of-the-art Deep Learning system(s) for Smart Data Web project • Learn from noisy text • Better approximate the highly non- linear arbitrary function • Pattern and Representation Learning, especially in Language Models with no explicit feature extraction Benefits of Deep Learning in NLP Extended Convolutional Neural Network2 Connectionist Bi-directional Neural Network2 Ranking Recurrent Neural Network (R-RNN)1 Entity/Concept Extraction Relation Extraction TriplesText (1) N.T.Vu, P. Gupta, H. Adel, H. Schütze. Bi-directional RNN with Ranking Loss for SLU. In ICASSP2016. (2) N.T.Vu, H. Adel, P. Gupta, H. Schütze. Combining Recurrent and Convolutional Neural Networks for Relation Classification. In NAACL2016. ( Siemens, Competitor-of, ABB ) Siemens, ABB Competitor-of
- 5. Restricted © Siemens AG 2016 31.05.2016Page 5 Corporate Technology Supervised Deep Learning in Joint Entity and Relation Extraction • Entity and relation inter-dependencies • Multi-tasking to jointly learn entity and relation representations and patterns • State-of-the-art system published3 for joint entity and relation extraction Motivation: Joint/Multi-task Learning Joint/Multi-task Neural Learning for End-to-End Entity and Relation Extraction Our State-of-the-Art system based on Neural Architectures for Joint Entity and Relation Extraction Neural Information Extraction System3 Text Documents (3) P. Gupta, A. Bernt, H. Schütze. Table Filling Multi-Task Recurrent Neural Network for Joint Entity and Relation Extraction. In COLING2016. Supplier-ofSupplier-of Supplier-of Competitor-of Sensor Sensor Sensor Sensor
- 6. Restricted © Siemens AG 2016 31.05.2016Page 6 Corporate Technology Deep Learning and NLP Applications at Siemens Public Domain: Web Semantic Search and Retrieval Application of Information Extraction in Public and Industrial domains Industrial Domain: Slot Filling for Product in Tender Documents Rectifier RATED CURRENT: ?? OUTPUT VOLTAGE: ?? OVERLOAD : ?? Query-Input Tender Documents, Service Reports IE System Query-Output Rectifier RATED CURRENT: 2666 A OUTPUT VOLTAGE: 1500 V OVERLOAD: 2 h
- 7. Restricted © Siemens AG 2016 31.05.2016Page 7 Corporate Technology Deep Learning and NLP Applications for TimeLines at Siemens Public Domain: TimeLine Generation from Biographies Application of Information Extraction in Public and Industrial domains Industrial Domain: TimeLine of Product for Historical Analysis and Monitoring (Future Work) Timeline of - ➢ Industrial products ➢ Industrial re-organization ➢ Business Strategy vs Profit Bloomberg-Biographies, European Data Forum Digitaleweltmagazin.de Intelligent TimeLine Extraction System