The WeKnowIt Project
2 likes545 views
The document discusses the concept of collective intelligence emerging from user-generated content, emphasizing the potential for deeper insights through data mining and analysis. It outlines various types of intelligence—personal, media, mass, social, and organizational—and provides examples and use cases in areas like emergency response and travel preparation. The authors highlight the challenges and opportunities in managing and extracting value from massive online activities, concluding that real-time applications and linked data represent future directions.
The WeKnowIt Project
- 1. Emerging, Collective Intelligence for Personal, Organisational and Social Use Symeon Papadopoulos, Yiannis Kompatsiaris (CERTH-ITI) www.weknowit.eu Trento, April 20 ICMR 2011
- 2. overview motivation & concept approach intelligence conclusions layers results
- 3. motivation • Users upload, tag, share, connect & search availability of massive amounts of user-generated content and data • Existing applications are limited to simple user data management or shallow analysis • Potential for much more if we mine the data and exploit them in the right context
- 4. collective intelligence …a form of intelligence emerging from online user activities Collective Intelligence >> sum of individuals’ intelligences
- 5. an example one of my photos @ flickr my location: N/A my tags: wki experiment bcn (…pretty uninformative)
- 6. an example one of my photos @ flickr others’ photos @ flickr tags location
- 7. an example alternative views / trends / facts one of my photos @ flickr
- 8. an example my friend’s photos @ flickr one of my photos @ flickr what did he visit next?
- 9. an example one of my photos @ flickr related Linked Data
- 10. collective intelligence @ weknowit personal intelligence media intelligence mass intelligence social intelligence organizational intelligence
- 11. overview motivation & concept approach intelligence conclusions layers results
- 14. media intelligence Visual Exploration
- 17. mass intelligence Tag Clustering
- 18. social intelligence Visualise Communities
- 19. social intelligence Community Browser
- 20. organisational intelligence Event-based Knowledge Sharing
- 21. organisational intelligence Distributed Group Management
- 22. architecture / integration Service Integration Knowledge and Content Storage Scenario-driven Service Composition
- 23. use case: emergency response Media Intelligence Personal Intelligence Photo arrives at ER control centre >> Automatic localisation of photo >> Login, Upload >> Photo & speech auto-tagging >> Spam detection >> Personalized Access Mass Intelligence >> Clustering >> Enrichment from additional sources Social Intelligence >> ER Alert Service >> Reputation Service Organisational Intelligence >> Log Merging & Viewing >> Incident Information Access
- 24. use case: travel Travel Preparation Mass Intelligence >> Landmark & Event detection >> Ranked facet lists of POIs >> Hybrid Image Clustering Media Intelligence >> Image Localisation >> Tag suggestions Mobile Guidance Personal Intelligence Post Travel >> Personal Recommendations Social Intelligence >> Group profiling & recommendations >> Friends position, alert
- 25. case: community detection in social media (1/2) • Structural similarity + Local expansion (highly efficient and scalable approach) • Not necessary to know the number of clusters • Noise resilient (not all nodes need to be part of a + community) • Generic approach adaptable to many applications (depending on node – edge representation) S. Papadopoulos, Y. Kompatsiaris, A. Vakali. “A Graph-based Clustering Scheme for Identifying Related Tags in Folksonomies”. In Proceedings of DaWaK'10, Springer-Verlag, 65-76
- 26. case: community detection in social media (2/2) PHOTOS & METADATA SPATIAL CLUSTERING + TEMPORAL ANALYSIS tags: sagrada familia, cathedral, barcelona taken: 12 May 2009 lat: 41.4036, lon: 2.1743 CLASSIFICATION TO LANDMARKS/EVENTS COMMUNITY DETECTION VISUAL TAG HYBRID S. Papadopoulos, C. Zigkolis, Y. Kompatsiaris, A. Vakali. “Cluster-based Landmark and Event Detection on Tagged Photo Collections”. In IEEE Multimedia Magazine 18(1), pp. 52-63, 2011
- 27. overview motivation & concept approach intelligence conclusions layers results
- 28. results: research • User modeling & interaction (CURIO, attention streams) • Media understanding (photo/text localization, photo/speech auto-tagging) • Media organization (graph-based clustering, faceted search, event detection) • Community analysis & management (administration, browsing, reputation, notification) • Knowledge representation & management (Event Model F, dgFOAF)
- 29. results: applications http://www.weknowit.eu/tr Integrated Prototypes • ER (desktop & mobile) • Travel (trip planning, mobile guidance, post-travel photo management) Stand-alone applications • WKI image recognizer • VIRAL (visual search and automatic localization) • ClustTour (city exploration by use of photo clusters) • Semaplorer++ • STEVIE (mobile POI management)
- 30. results: exploitation VIRAL evaluation by Vodafone 360
- 31. results: public APIs http://mklab.iti.gr/wki-apps
- 32. conclusions ...so far • CI emerges from massive online activities • it is hard to extract and manage • ...but is definitely worth the effort. in the future... • other domains: news, finance, e-gov • real-time CI • CI Linked Data
- 33. thank you! Presentation online @ http://www.weknowit.eu > news
- 35. content in weknowit offline model creation, training Standard annotated corpora used for training. • Single-modality: text (Brown corpus), speech (TIMIT database), standard training data image (Corel database) • Single-source: prepared by a single person/organization • Consistent quality: absence of spam, malicious or erroneous data • Small-moderate volume: Manually produced Massive user generated content and feedback from Web 2.0 applications • Multi-modality: e.g. image + tags, image + geo-location + time massive Web 2.0 • Multi-source: may be generated by different applications, user communities, e.g. Flickr, Panoramio, PhotoBucket • Inconsistent quality: noise, spam, ambiguity • Huge volume: Massively produced and disseminated online user profiling, method invocation Online content and user actions by WeKnowIt users. It is mainly used for WKI user-contributed triggering WeKnowIt services and for providing context to them, e.g. user profile, input content to be used as example for querying, etc.
- 36. technical approach Variety of approaches depending on content-metadata input. massive Web 2.0 – massive Web 2.0 – standard training data semi-structured unstructured WKI user-contributed standard Statistical approaches Content analysis Knowledge Based Probabilistic models Text models (n-gram, LDA, CRF) Lookup (WordNet) (pLSA, Bag-Of-Words) Image processing Thesaurus Lookup (GeoPlanet) Graph-based approaches (visual feature extraction) Concept detection (SNA, community detection) Speech modeling (Wikipedia, domain (spectral analysis, HMM) ontologies)
- 37. massive Web 2.0 WKI user-contributed standard training data Locations Topics Social connections WP1 Get recommendations Tag normalization Emergency alert service Tag processing WP4 Visual analysis WP2 Community analysis tool WP2 Text classification Text annotation ClustTour Events POI recommendation WP3 Speech search Local tag community detector WP2 WP3 POI clustering Semantic photo query Search place POI Entities Named entity detection Log merger WP5 csxPOIs WP3 WP5 Entity facet extraction - ranking Semaplorer(++) Representation Access CURIO Storage Account Manager WP1 WP1 VERACITY Login WP2 Speech Indexing WP5 Event model F + M3O WP4 Community administration platform WP6 Data Storage WP6 Common data model WP5 Group Management GUI ER CSG WP6 Mobile app Travel preparation Manage Item Comment System Integration WP1 Tag Users messaging Desktop proto WP7 Mobile guidance WP3 Search Knowledge Base Lexical Spam Detector Post ER tool Post-travel logging
- 38. weknowit work structure WP9: Management management WP1: Personal Intelligence WP6: Architecture / Integration WP2: Media Intelligence WP3: Mass Intelligence WP7.I Use Case: ER WP4: Social Intelligence WP7.II Use Case: Travel WP5: Organisational Intelligence research development WP8: Dissemination & Exploitation dissemination & exploitation
- 39. Causality Pattern in Event-Model-F • Event (cause) implies other event (effect) • Causal relationship holds under some justification • Causes and effects are events, and only events
- 40. OntoMDE • Specification of MoOn using eCore and OAM as UML2 class diagram • Transformation steps implemented • Evaluation with ontologies of different complexity
- 41. Content vs. Structure Concepts
- 42. results: dissemination Activities • Collective Intelligence Workshops and Special Session • Summer schools Publications • 8 journal publications • Trans. on MultiMedia, IEEE MultiMedia, J. of Web Semantics, MTAP, etc. • 59 conference papers • ACM MultiMedia, SIGIR, CVPR, ESWC, WWW, ICIP, WSDM, etc. • 2 CI book chapters + 1 CI White Paper • 3 patent applications