Comparing Ontotext KIM and Apache Stanbol
Download as PPT, PDF8 likes2,940 views
Ontotext KIM and Apache Stanbol are compared for their abilities to semantically annotate documents. KIM provides more extensive annotations including entities, relations, dates and numbers, achieving 79% recall and precision. Stanbol annotations have lower 47% recall and 50% precision due to text analysis problems and a smaller knowledge base. While Stanbol shows promise, its semantic processing lags behind established frameworks like KIM.
![KIM Annotations KIM annotates: organizations , persons , positions , locations , general terms , time , years , numbers Hover over an annotation to see its type Click to see entity description from World KB Click [+] to see more details Click [D] to see related documents Even finds relations: Lisa Whatmough , founder of Squint Ltd Trend Bible founder Joanna Feeley Recall and precision are both quite good! But not perfect, e.g. : the Queen ’s Diamond Jubilee [Place]: should be [Time] like Golden Jubilee Dahlgren [Company]: should be [Person] as in Maria Holmer Dahlgren but that's in the previous sentence Tent London [Country Capital]: is actually an event ( design trade show ) London Design [Organization] Festival : is actually an event (festival) # Sep 2011 Comparing KIM and Stanbol](https://image.slidesharecdn.com/comparingontotextkimandapachestanbol-110912121842-phpapp01/85/Comparing-Ontotext-KIM-and-Apache-Stanbol-8-320.jpg)
![Stanbol Annotation Go to Stanbol Demo , paste document text from LatestNews, click [Run Engines] # Sep 2011 Comparing KIM and Stanbol](https://image.slidesharecdn.com/comparingontotextkimandapachestanbol-110912121842-phpapp01/85/Comparing-Ontotext-KIM-and-Apache-Stanbol-9-320.jpg)
Comparing Ontotext KIM and Apache Stanbol
- 1. Vladimir Alexiev, PhD, PMP Comparing Ontotext KIM and Apache Stanbol
- 2. Presentation Outline What is Ontotext KIM? What is Apache Stanbol? KIM Showcases: Latest News, Exopatent KIM-annotated Document Stanbol-annotated Document Comparison and Conclusions # Sep 2011 Comparing KIM and Stanbol
- 3. What is Ontotext KIM? KIM is a product of Ontotext, provider of core semantic technologies Long-established Semantic Annotation and Search platform (over 6 years) Based on the open-source GATE platform (General Architecture for Text Engineering) that is not just established but entrenched (16 years) # Sep 2011 Comparing KIM and Stanbol
- 4. KIM Showcases KIM Showcases include two live annotation demos: Latest News (general news stream), including a World KB of some 500k entities Exopatent (drug patents: complex domain and relations) # Sep 2011 Comparing KIM and Stanbol
- 5. What is Apache Stanbol? Currently in incubation at Apache Foundation Part of the EU research project IKS (Interactive Knowledge Stack) 4 years (2009-2012), 6.6 MEUR co-funding Open source modular software stack and reusable set of components for semantic content management Focused on building a flexible technology platform for semantically enhanced Content Management Systems IKS provides 40 Early Adopter grants (5k-7k EUR) to CMS willing to integrate Stambol Integrates to Nuxeo and 5 other CMS 9 more contracts are signed, 22 more proposals received Implements 3 and plans 6 Services Implements 3(?) and plans 7 Enhancement (annotation) Engines Entities : small index of approx. 43k dbpedia entities comes with the default installation. # Sep 2011 Comparing KIM and Stanbol
- 6. Document to Annotate (from LatestNews ) Let's give it a try! Click on a random document in LatestNews Document metadata is extracted by KIM and includes: As you will see on next page, Key Phrases and Entities are extracted quite precisely # Sep 2011 Comparing KIM and Stanbol Date 08-09-2011 Title Capital trend that just won’t die Source The Independent Language english URL http://rss.feedsportal.com/c/266/f/3817/s/181af8b8/l/0L0Sindependent0O0Clife0Estyle0Chouse0Eand0Ehome0Cinteriors0Cannie0Edeakin0Ccapital0Etrend0Ethat0Ejust0Ewonrsquot0Edie0E23513230Bhtml/story01.htm Key Phrases designer, trend, nostalgia, tray, mania trend Key Entities the Queen, Queen Elizabeth, Misha Black, Lisa Whatmough, Annie Deakin, Barbara Chandler, Tate Modern, Maria Holmer Dahlgren, St.Paul, Kate Adams, Joanna Feeley, Squint Ltd, London Design, London Transport Museum, London
- 7. KIM Annotated Document # Sep 2011 Comparing KIM and Stanbol How long can the Brit mania last? Rather a long time, insist trend predictors and designers showing at this month ’s interior exhibitions. With Top Drawer starting this Sunday and London Design Festival just a few weeks away, the capital city is awash with iconic – and subtle - London imagery. The longevity of this Brit mania trend , it seems, lies in the originality of the designers . Instead of plastering the Union Jack or tube map prints onto everything, designers realize an urgency for creativity. Expect to find innovative twists on the London trend ; a sofa upholstered in that furry seating fabric in the Tube, trays decorated with raining cats and dogs and bulldog embossed wallpaper. Last year , a micro trend for all things London was obvious at Top Drawer, a trade show for design-led gifts and this year , sees little difference. ‘With the arrival of the Olympics and the Golden Jubilee in 2012 , we’re celebrating all things London at Top Drawer this autumn ,’ say the organizers of Top Drawer. Back in 2009 , market forecasters Trend Bible announced that the London and transport trend would be a long-term keeper. They initially flagged up the trend of transport and nostalgia for British icons in their Voyager trend showcased in their Spring / Summer 2011 Home Trends book written back in 2009 . ‘Many of our clients have had real commercial success with a type of British nostalgia-whether it is Union Jack cushions or vintage Queen Elizabeth photographic style prints,’ says Trend Bible founder Joanna Feeley . ‘But really the question they are all asking is how they can keep this look fresh and move it forward, since British nostalgia as a trend concept continuing to be important through 2011 and into 2012 with the Queen ’s Diamond Jubilee and London Olympics still to come next summer .’ The secret lies in originality and an aversion to splattering the Union Jack or tube map onto furnishings. In response to the tacky souvenirs and cheap throwaway London designs for tourists, Swedish designer Maria Holmer Dahlgren wanted to celebrate the city using contemporary graphics. The result is her London collection, which will be on show at Top Drawer this weekend . It comprises trays , mugs and aprons adorned with the Tate Modern , Brick Lane and Tower Bridge. Humour is key to her success; Dahlgren epitomizes well-known British idiosyncrasies as she pictorially sums up our wet weather with cats and dogs falling under an umbrella. Also showing at Top Drawer is ceramicist Kate Adams , of mydeco design boutique, who spent five years at Cockpit Arts where she established her London skyline tableware range. Each piece is thrown on the potter’s wheel, then individually illustrated with rugged versions of iconic buildings such as St.Paul’s Cathedral and the Gherkin.
- 8. KIM Annotations KIM annotates: organizations , persons , positions , locations , general terms , time , years , numbers Hover over an annotation to see its type Click to see entity description from World KB Click [+] to see more details Click [D] to see related documents Even finds relations: Lisa Whatmough , founder of Squint Ltd Trend Bible founder Joanna Feeley Recall and precision are both quite good! But not perfect, e.g. : the Queen ’s Diamond Jubilee [Place]: should be [Time] like Golden Jubilee Dahlgren [Company]: should be [Person] as in Maria Holmer Dahlgren but that's in the previous sentence Tent London [Country Capital]: is actually an event ( design trade show ) London Design [Organization] Festival : is actually an event (festival) # Sep 2011 Comparing KIM and Stanbol
- 9. Stanbol Annotation Go to Stanbol Demo , paste document text from LatestNews, click [Run Engines] # Sep 2011 Comparing KIM and Stanbol
- 10. Stanbol Annotations Stanbol uses the following Enhancement Engines: NamedEntityExtraction, NamedEntityTagging, CachingDereferencer You can also select the Output format (e.g. JSON-LD, Turtle..) to see technical details and the way text is parsed Doesn't show the annotations in context Recognizes only Entities, not relations, dates, numbers, general concepts Shows a map of recognized locations at the bottom Recall is much lower than KIM, which is no wonder since Stanbol is seeded with a small KB from dbpedia But precision is just horrendous! (see further) # Sep 2011 Comparing KIM and Stanbol
- 11. Stanbol Precision Stanbol Precision is horrendous. Text analysis problems: Text mangling: "St.Paul’s Cathedral" is parsed as "St.Paul s Ca l" (why chars are replaced with spaces ??) which leads to identifying "Ca" as a place. But the article does not mention California even once! Sentence segmentation: "Barbara Chandler . Her": why "Her" from next sentence is tacked to this entity? Incomplete matching: took only the bold words from "Love London " (a book), "London Transport Museum " (an organization) Missed co-reference: "Whatmough" not recognized the same as "Lisa Whatmough" NamedEntityTaggingEngine makes up facts trigger-happily: Silver Spring, Maryland from "Spring/ Summer 2011" Union Pacific Railroad, Auto Union and International Astronomical Union (!?) from " Union Jack " (the English flag) Royal Marines, Royal Navy, Royal Air Force from " the Royal wedding " Wrong entity identification: "District Line" and "Green Line" are not Organizations but subway lines "Humour" is not an Organization but a word # Sep 2011 Comparing KIM and Stanbol
- 12. Comparison of Annotations KIM: Person : Queen Elizabeth = the Queen , Joanna Feeley , Maria Holmer Dahlgren , Annie Deakin , Barbara Chandler , Misha Black , Lisa Whatmoug h= Whatmough , Kate Adams Wrong : Tate Modern Organization : Trend Bible , Squint Ltd , London Transport Museum Wrong : London Design Festival , Dahlgren Place/Facility : London , Regent Street Wrong : Diamond Jubilee Position : founder General concept : designers , trend , nostalgia , tray ( s ), mania trend Time reference : this month , this Sunday , Last year , this year, Golden Jubilee , Spring , Summer , this autumn , next summer , 17-25 September , this weekend , five years , 100 years Year : 2008 , 2009 , 2011 , 2012 Stanbol: Person : Kate Adams, Lisa Whatmough, Maria Holmer Dahlgren, Misha Black Wrong : Barbara Chandler . Her Organization : Cockpit Arts, Conran Shop, Transport Museum, Squint Ltd Wrong : District Line, Green Line, Humour, Royal Navy, Union Pacific Railroad Wrong (lower confidence) : Auto Union, International Astronomical Union, Royal Marines, Royal Air Place : London Wrong : Love London, Ca, Silver Spring Maryland # Sep 2011 Comparing KIM and Stanbol
- 13. Comparison of Recall and Precision We compare only Person + Organization + Place KIM also annotates Position, General concept, Time reference, Year, Number KIM Recall: 15/19= 79% found 10+3+2=15 correct (including2 co-references) missed 4 orgs (the org missed in "market forecasters Trend Bible" but found in "Trend Bible founder Joanna Feeley") Precision : 15/19= 79% found total 11+5+3=19, wrong 1+2+1=4 Stanbol Recall: 9/19= 47% found 4+4+1=9 correct Precision: 9/18= 50% found total 5+9+4=18, wrong 1+5+3=9 If lower confidence mis-hits are taken into account: 9/22= 41% # Sep 2011 Comparing KIM and Stanbol
- 14. Conclusions Stanbol is a promising open source project that may bring semantic technologies to mass-market CMS systems Another similar project is SCMS (Semantic Content Management Systems for Enterprise Knowledge Management & News Mining) funded under the Eureka EuroStars program Stanbol creates useful research and training materials: E.g. paper A Semantic Backend for Content Management Systems E.g. training presentation Semantifying Your CMS Stanbol may establish a " reference architecture " for integrating CMS to semantic technology components (e.g. CMS Adapter component, FactStore API, CMIS API…) However, semantic content processing in Stanbol is still far behind established text analysis frameworks # Sep 2011 Comparing KIM and Stanbol