HistoGraph presentation Insa de Lyon
Download as PPTX, PDF2 likes471 views
The document discusses the CUbRIK project, which aims to build a social graph from image archives by reconstructing and exploring social ties through historical sources. It involves four pillars: close collaboration with researchers, a structured repository of historical persons and events, efficient face identification indexing, and analysis/visualization tools. Challenges include determining the truth given differing humanities and computer science perspectives, engaging crowds, and addressing image copyrights. The project seeks to refine its tools based on researcher requirements to better analyze relationships and social networks over time from multimedia sources.
HistoGraph presentation Insa de Lyon
- 1. histoGraph Building a Social graph from image archives La science et les effets de réseau, Lyon 24.03.2014 www.cubrikproject.eu Lars Wieneke, CVCE, Luxembourg 0
- 2. The CVCE Lars Wieneke, CVCE, Luxembourg 1
- 3. About CUbRIK European Community's Seventh Framework Program FP7-ICT 15 European partners Multimedia search processing: Putting humans in the loop Demos: History of Europe and Fashion Lars Wieneke, CVCE, Luxembourg 2
- 4. Point of departure Images as sources Lars Wieneke, CVCE, Luxembourg 3
- 5. Goal: Reconstructing and exploring social ties through historical sources Lars Wieneke, CVCE, Luxembourg 4
- 6. Towards the social graph 4 pillars 1. Close connection to the requirements of researchers in European Integration studies 2. Structured and referencable repository of persons, events and places in time 3. Efficient indexation process that enables the association of faces with identities 4. Toolchain for analysis and visualization Lars Wieneke, CVCE, Luxembourg 5
- 7. Towards the social graph Sourcing researcher requirements Lars Wieneke, CVCE, Luxembourg 6 Selection of target user group First draft of the app scenario Feedback on technical scope Exploratory interviews (daily work practices) Second draft of the app scenario Focus group (user needs and app scenarios) Feedback on technical feasability Lessons learned: issues and features Specification Implementation 1. demonstrator Workshop: Review of app and features Revised specification Implementation 2. demonstrator Evaluation and test Stage 1 Stage 2 Stage 3 Stage 4 Stage 5 Users Requirements Technology
- 8. Towards the social graph Structured repository Lars Wieneke, CVCE, Luxembourg 7
- 9. Towards the social graph Indexation process 8 Raw content High level features (automatic annotations) Conflict (e.g., “Image contains ‘Romano Prodi’ ” ? Confidence = low) Conflict store Conflict manager Conflict resolution task store Conflict resolution task: conflict, required skill, priority, .. CUbRIK app for Conflict resolution Game Crowdtask Q&A Lars Wieneke, CVCE, Luxembourg
- 10. 9 Towards the social graph Indexation process II Human in the loop added value: Verification of identities/places/events ambiguous and temporal only possible by putting humans in the loop Integration of multiple perspectives CUbRIK as an open toolbox allows follow-up and extension through third parties “Vertical” integration: GUI, components, crowdsourcing integrated in a platform Lars Wieneke, CVCE, Luxembourg
- 11. 10 Towards the social graph Visualization and analysis Lars Wieneke, CVCE, Luxembourg
- 12. 11 Challenges & Approach Main challenges Detection and identification of identities/places/events in time Verification of identities/places/events in time Analysis of relationships (e.g. co-occurrences) Rights aware crawling and storage Verification of provenance and license information Truth and provenance Approach Crowd-sourced verification of detected faces (false positives/negatives) Verification of identities through/places/events in time social networks of experts Visual knowledge discovery/exploration Integrated rights aware crawling and storage Integrated license and provenance management 27/11/2013 Lars Wieneke, CVCE, Luxembourg
- 13. Towards the social graph Bringing it all together Social Graph Network Analysis CROWD Research Inquiry Graph Visualizat ion Lars Wieneke, CVCE, Luxembourg 12 Image Indexation Media harvest ing and upload Face detect ion Face ident ifi cat ion Clickworkers Crowd Face posit ion validat ion Copyright aware crawler Provenance checker License checker Content provider tools Metadata Ent ity extract ion Identity reconciliation Ent ity verifi cat ion & annotat ion Ent itypedia Integrat ion CROWD pre-fi ltering Text Indexation Connect ion to the CVCE collect ion Ent ity anntat ion and extract ion Expert Crowd Expert CROWD verifi cat ion Ent itypedia Integrat ion CROWD Research Inquieries Expert Crowd Graph Visualizat ion Analysis of the social graph Graph Query (old: Query for Entities) Graph Visualizat ion Query for ent it ies Context Expander Expansion through documents Expansion through videos Expansion through images Expansion through related ent it ies Social Graph construction Social Graph Creat ion Content Analysis and Enrichment Querying Feedback acquisition and processing Y3 component Query for spat ial constraints EXP through SIMILAR images WP Event detect ion
- 14. histoGraph demo Time for a Demo! Lars Wieneke, CVCE, Luxembourg 13
- 15. Pipelining the CUbRIK components: Human input from click-workers Great choice for simple tasks: Face detection: false positives, false negatives Monetary motivation, via www.microtask.com Poor performance on complex tasks: Low resolution images Different angles etc. Actors recurring over time Lars Wieneke, CVCE, Luxembourg 14
- 16. Pipelining the CUbRIK components: Human input from experts Capable of complex tasks: In-depth knowledge of key actors Context knowledge allows inferences But: Different motivational models! Public goods Reputation Lars Wieneke, CVCE, Luxembourg 15
- 17. Usage for historians No one truth in history but interpretation, context and discussion Therefore need to represent ambivalence, contradictions and discussion Close ties between data representation (Social graph) and their original context (primary sources) Lars Wieneke, CVCE, Luxembourg 16
- 18. Conclusion Challenges What is truth? Humanities vs. Computer Science Gathering requirements for tools that haven‘t been developed yet Engaging crowds Image copyrights Scientific value? Refinement of the application Additional datasources Improvement of the interface Integration of the new components Lars Wieneke, CVCE, Luxembourg 17
- 19. Outlook CUbRIK Presentation 18
- 20. Outlook CUbRIK Presentation 19
- 21. Outlook histoGraph CUbRIK Presentation 20
- 22. Visit us on WWW.CUBRIKPROJECT.EU @CUBRIKPROJECT CUbRIK Presentation 21 Or follow us on Twitter
Editor's Notes
- #2: Ok, let’s start with a joke to relax you guys a bit: How can you tell that a computer scientist received communications training?
- #11: Kern des Problems: keine automatische Funktionalität, daher CUbRIK Ansatz mit Humans in the loop Dann Bullet: open toolbox Integration verschiedener Modelle Auf Grundherausforderung beziehen Nicht nur auf Personen beziehen, allgemeiner auf Events/Orten Auf Grundlage der CUbRIK Entwicklungen… Weiterentwicklung! Neue Ansätze durch Dritte
- #12: Kern des Problems: keine automatische Funktionalität, daher CUbRIK Ansatz mit Humans in the loop Dann Bullet: open toolbox Integration verschiedener Modelle Auf Grundherausforderung beziehen Nicht nur auf Personen beziehen, allgemeiner auf Events/Orten Auf Grundlage der CUbRIK Entwicklungen… Weiterentwicklung! Neue Ansätze durch Dritte
- #13: Technical challenges: Vielleicht doch nach Bedeutung sortieren! Auf das gesamte Bild beziehen, Ausblick integrieren. Noch zu stark auf Identities ausgerichtet, was kann man damit machen, wenn man diese hat? Visual knowledge discovery/exploration