Maastricht university - Text-Mining: Big Data Analytics voor ongestructureerde data
- 1. Text-Mining: Big Data Analytics voor ongestructureerde data Prof dr ir Jan C. Scholtes
- 3. 3 Text-Mining: structuring the unstructured
- 4. 4 Text Mining Text Mining: The next step in Search Technology Finding without knowing exactly what you’re looking for, or finding what apparently isn’t there.
- 5. 5 5 •Social network analysis •Community Detection •Different types of visualization for temporal, geographical, semantic or relational mappings. •Decision Tree •Bayes Classifiers •Rochio •k-NN •Support Vector Machines •Clustering •Entity extraction •Fact, Event & Concept extraction •Negations, co-reference resolution •Grammars •Statistical methods: Hidden Markov Models, Maximum Entropy Models, Conditional Random Fields, … •Data normalization (Ontology matching) •Inverted file index •Relevance ranking •Relevance feedback •Faceted search •Incomplete matching •Index compression •Precision & Recall Search Information Extraction Link Analysis & Data Visualization Machine Learning
- 6. 6 Language_Name English CITY New Brunswick, WASHINGTON COMPANY J&J, Johnson & Johnson COUNTRY Greece, Poland, Romania, United Kingdom CURRENCY .02 USD, 21400000 USD, 48600000 USD, 59.47 USD, 70000000 USD DATE 04-08 DAY Fri, Friday NOUN_GROUP biotech drugs, bribery case, denying guilt, final growth frontier, foreign countries, giving gifts, holding corporations, intense revenue pressure, meaningful credit, medical device kickbacks, medical devices, multiple businesses, next several days, non-U.S. markets, only way, orthopedic hips, other countries, over-the-counter medicines, paid kickbacks, past year, paying kickbacks, same time, several new positions, similar violations, travel gifts ORGANIZATION Department of Justice, Justice Department, SEC, Securities and Exchange Commission, University of Michigan PEOPLES Iraqi PERSON Erik Gordon, Mythili Raman, William Weldon PLACE_REGION Europe PRODUCT Benadryl, Tylenol PROP_MISC Band-Aids, Food Program, Foreign Corrupt Practices Act, United Nations Oil STATE N.J. TIME 1:32 pm ET TIME_PERIOD 13 years, five years, six months, three years YEAR 2007 Problem "We went to the government to report improper payments and have taken full responsibility for these actions," said William Weldon, Chairman and CEO of J&J., Last month federal health regulators took legal control of the plant where millions of bottles of defective medication were produced., The charges against J&J were brought under the Foreign Corrupt Practices Act, which bars publicly traded companies from bribing officials in other countries to get or retain business., The company will pay $21.4 million in criminal penalties for improper payments and return $48.6 million in illegal profits, according to the government., The SEC says J&J agents used fake contracts and sham companies to deliver the bribes. Sentiment giving meaningful credit to companies that self-report, We are committed to holding corporations accountable for bribing foreign officials, what is honest Request make sure it complies with anti-bribery laws across its businesses
- 10. 10 WHY & HOW: Emotion Detection
- 12. 12 Text Mining the Lord of the Rings • Automatic identification of key players (custodians) • Automatic identification of locations. • Automatic identification of travel patterns of key players. • Visualize in time.
- 14. Memory Consistency 24/7 Speed & Scalability Search M&A and Restructuring Data Collection Analytics eDiscovery, Regulatory Requests, Investigations, Fact Finding Missions Reporting Archiving Knowledge Management Production Big Data Analytics and the Law
- 15. SLIDE / 15
- 16. SLIDE / 16 Source: Comparing the Performance of Artificial Intelligence to Human Lawyers in the Review of Standard Business Contracts, February 2018, LawGeex.
- 17. • Lack of precision leads to noise, too many false hits, too much work to review, which yields high cost of review. • Lack of recall leads to missing relevant documents which yields risk. 17
- 18. 18 Human Performance • When both precision and recall are over 80%, human performance is approached. • This applies to the best humans. • It can be argued that values over 80% are often subject to different interpretations and discussions. 18
- 19. 19 eDiscovery, Fact Finding Missions (waarheidsvinding), Investigations (regulatory and internal), Evidence Seizure (bewijsbeslag), …
- 20. Teaching the computer what you are looking for
- 21. 21 Results • Find more 2-3x relevant documents • In fraction of the time to review entire data set • You know exactly what percentage of relevant documents you found • No need to understand complex search tools or queries: just reviewing
- 22. SLIDE / 22
- 23. Thank you! Time for Q&A Prof dr ir Jan C. Scholtes https://www.linkedin.com/in/jscholtes/