ECM Meets the Semantic Web - Nuxeo World 2011
2 likes1,561 views
The document discusses open source enterprise content management and how it can be enhanced by integrating semantic web technologies. It describes how semantic technologies can help extract meaning from unstructured content, connect information to form knowledge, reason about the knowledge, and present it in an actionable way. The document also provides an overview of Nuxeo's work on semantic ECM through various research projects and their semantic engine which extracts metadata from content.
ECM Meets the Semantic Web - Nuxeo World 2011
- 1. When ECM Meets the Semantic Web 20 Oct 2011 - Olivier Grisel & Stefane Fermigier Open Source ECM Thursday, October 20, 2011
- 2. Business Motivations 2 Thursday, October 20, 2011
- 3. Source: Wikipedia Thursday, October 20, 2011
- 4. Source: Wikipedia Thursday, October 20, 2011
- 5. The DIKW hierarchy 5 Thursday, October 20, 2011
- 6. But every coin has another side Thursday, October 20, 2011
- 8. A few figures • 50% more data / content / information produced every year • 1.8 zettabytes of data produced in 2011 (= 1 billion terabytes) • Employees are drowning in a sea of email, status messages, etc., and spend on average more than 6 hours / weeks unsuccessfully searching for or recreating lost documents Thursday, October 20, 2011
- 9. A Solution: the Semantic Web 9 Thursday, October 20, 2011
- 10. A Brief History of the Web • Web 1.0 (1990-now): web of sites and pages, aka the World Wide Web • Web 2.0 (2000-now): web of people and of participation, aka the Social Web (Blogs, RSS, tags, Facebook, Wikipedia, etc.) • Web 3.0 (2010-now): web of data, of meaning and connected knowledge, aka the Semantic Web 10 Thursday, October 20, 2011
- 11. 11 Thursday, October 20, 2011
- 12. “To a computer, then, the web is a flat, boring world devoid of meaning” Tim Berners Lee, http://www.w3.org/Talks/WWW94Tim/ 12 Thursday, October 20, 2011
- 13. “This is a pity, as in fact documents on the web describe real objects and imaginary concepts, and give particular relationships between them” Tim Berners Lee, http://www.w3.org/Talks/WWW94Tim/ 13 Thursday, October 20, 2011
- 14. “Adding semantics to the web involves two things: allowing documents which have information in machine-readable forms, and allowing links to be created with relationship values.” Tim Berners Lee, http://www.w3.org/Talks/WWW94Tim/ 14 Thursday, October 20, 2011
- 15. “The Semantic Web is not a separate Web but an extension of the current one, in which information is given well-defined meaning, better enabling computers and people to work in cooperation.” Tim Berners Lee, http://www.w3.org/Talks/WWW94Tim/ 15 Thursday, October 20, 2011
- 16. Means and Tools 16 Thursday, October 20, 2011
- 17. 4 stages • Extract meaning from raw data / content • Connect information to form knowledge • Reason about this knowledge • Present this knowledge in actionable form 17 Thursday, October 20, 2011
- 18. Extracting • Leverage metadata embedded in or associated with documents (when they exist) • Or use machine learning, NLP (Natural Language Processing) and image processing algorithms to extract meaning from text / images • Examples include: named entities extraction, automatic categorization / tagging, sentiment analysis, etc. 18 Thursday, October 20, 2011
- 19. Interlude: Linked Open Data 19 Thursday, October 20, 2011
- 20. 2007 2008 2009 2010 20 Thursday, October 20, 2011
- 21. 2011! 21 Thursday, October 20, 2011
- 22. Linking • Many Linked Open Data repositories have been made available over the last 10 years • RDF and graph database systems are now available to manage this huge mass of information (billions of triples) • Match information extracted from content with these public (or internal) data/knowledge bases 22 Thursday, October 20, 2011
- 23. Reasoning • When you are working on reliable metadata (ex: RDFa embedded in web pages), you can use rule / inference engines to infer actionable knowledge from your content (ex: shopping recommendation engine) • Rules can also be used to clean up / flag errors when working with unreliable (e.g. automatically extracted) information 23 Thursday, October 20, 2011
- 24. Presenting • Allow the users of your system to interact with the knowledge thus extracted or produced, in a way that allows them to do their jobs better • A smart presentation system solves the information overload issue by contextualizing the information, i.e. presenting only information relevant to what the user is currently doing 24 Thursday, October 20, 2011
- 25. R&D Projects Involving Nuxeo 25 Thursday, October 20, 2011
- 26. IKS project • European R&D project under the FP7, with 13 partners (6 SMEs) and a 8.5M EUR budget • Goal: create a semantic software “stack” that will be used by CMS vendors to add semantic features to their products • Started in Jan. 2009, will last until Dec. 2012 • First tangible result: Apache Stanbol (more about this later) 26 Thursday, October 20, 2011
- 27. SAMAR project • French collaborative R&D project with 10 partners, and a 4.5M EUR budget • Goal: create a platform for managing multimedia content in arabic, for news agencies such as AFP • Will include: automated translation, named entities extraction, content classification • First results: integration between Nuxeo and Temis (more later) 27 Thursday, October 20, 2011
- 28. State of the Art Semantic ECM at Nuxeo 28 Thursday, October 20, 2011
- 29. The Semantic Engine • From unstructured content to Knowledge • Language guessing • Topic classification (Business, Sports, Media, ...) • Named Entities extraction and linking • Relationships and properties extraction 29 Thursday, October 20, 2011
- 30. Demo time! 30 Thursday, October 20, 2011
- 31. 31 Thursday, October 20, 2011
- 32. 32 Thursday, October 20, 2011
- 33. 33 Thursday, October 20, 2011
- 34. RESTful is Beautiful 34 Thursday, October 20, 2011
- 35. 35 Thursday, October 20, 2011
- 36. 36 Thursday, October 20, 2011
- 37. = Semantic Engines (Apache OpenNLP) + Fast Linked Data local index (Apache Solr) + Semantic Rule Engine 37 (Apache Jena) Thursday, October 20, 2011
- 38. Apache Stanbol Engine 1 DBpedia Engine 2 2 1 Engine 3 Freebase Nuxeo DM 3 addon Geonames LDAP Local IT infrastructure (LAN) 38 Thursday, October 20, 2011
- 39. How to build engines? 39 Thursday, October 20, 2011
- 40. Training statistical models for NER with Wikipedia and DBpedia • Extract sentences with link positions in Wikipedia articles • DBPedia to the find type of the target entity (Person, Location, Organization) • Apache Pig scripts to compute the join + format the result as training files for OpenNLP • Apache OpenNLP to build and evaluate the models • Apache Hadoop for distributed processing • Apache Whirr for deployment and management on Amazon EC2 cluster 40 Thursday, October 20, 2011
- 41. 41 Thursday, October 20, 2011
- 42. 42 Thursday, October 20, 2011
- 43. 43 Thursday, October 20, 2011
- 44. 44 Thursday, October 20, 2011
- 45. Training statistical models for topic classification from Wikipedia and DBpedia • Filter category tree from DBpedia SKOS entries (~500k) • Pig scripts to compute the joins with articles abstracts for all the articles categorized in Wikipedia • Export as 2.8GB TSV file to be indexed in Apache Solr • Use Solr MoreLikeThisHandler to find the top 3 most related Wikipedia category for any kind of text • Apache Whirr & Hadoop for deployment and management on Amazon EC2 cluster 45 Thursday, October 20, 2011
- 46. Wrap Up on Recent Work • Full offline mode: Stanbol EntityHub • Multi-lingual Indexes • New UI for occurrences reviews • Temis Luxid Annotation Factory integration 46 Thursday, October 20, 2011
- 47. What’s next? • Stanbol and Temis connection in Admin Center • Embedded Stanbol mode for easy deployment • More OpenNLP models for more languages • Finalize topic classification - handle hierarchy • Tight integration with Nuxeo DM search features 47 Thursday, October 20, 2011
- 48. Thank you for your attention! 48 Thursday, October 20, 2011