Visualizing Provenance using Comics

Visualizing Provenance
using Comics
Andreas Schreiber
German Aerospace Center (DLR)
Regina Struminski
University of Applied Science Düsseldorf

> TaPP'17 > Andreas Schreiber, Regina Struminski • Visualizing Provenance using Comics > 23.06.2017DLR.de • Chart 2
Introduction
Simulation and Software Technology, Cologne/Berlin
Head of Intelligent and Distributed Systems department
Institute of Data Science, Jena
Head of Secure Software Engineering group
Co-Founder
Data Scientist
Patient

Motivation – Use Cases
Quantified Self (n = 1 participant) Medical Trials (n > 1 participants)

Motivation – Use Cases
Telemedicine Medical experiments

Understand, how Quantified Self data has been produced, processed,
stored, accessed, …
Pictures from Breakout Session on Mapping Data Access (2014 QS Europe Conference, Amsterdam)
https://forum.quantifiedself.com/t/breakout-mapping-data-access/995

Example: Weight Tracking Workflow

Questions related to Quantified Self Data and Activities
Data
• What data about the user were created during the activity X?
• What data about the user were automatically generated?
• What data about the user were derived from manual input?
Apps and Services
• Which activities support visualization of the users data?
• In which activities can the user input data?
• What processes are communicating data?
Access and Privacy
• What parties were involved in generating data X?
• What parties got access on data X?
• Can other parties see user’s data X?

Provenance Model for Quantified Self
Sub models for basic Activities
• Input
• Sensing
• Export
• Request
• Aggregate
• Visualize
The activities generate or change data
that is associated or attributed to Agents
• Users
• Software
• Organizations
• Schreiber, A. (2016) A Provenance Model for Quantified
Self Data. In: Universal Access in Human-Computer
Interaction. Methods, Techniques, and Best Practices: 10th
International Conference, UAHCI 2016, Held as Part of HCI
International 2016, Toronto, ON, Canada, July 17-22, 2016,
Proceedings, Part I, Springer, 382-393
• Schreiber A., Seider D. (2016) Towards Provenance
Capturing of Quantified Self Data. In: Provenance and
Annotation of Data and Processes. IPAW 2016. Lecture
Notes in Computer Science, vol 9672. Springer, Cham

UserData
Input
User
wasGeneratedBy
wasAssociatedWith
wasAttributedTo
prov:startTime
prov:endTime
prov:type
prov:type
prov:label
prov:time
Software
type= prov:SoftwareAgent
prov:label
wasAssociatedWith
type=prov:Person
prov:label

UserData
Visualize
User
Graphic
used
wasGeneratedBy
wasDerivedFrom
type=prov:Person
prov:label
wasAttributedTo
prov:type
prov:label
prov:type
prov:label
prov:time
prov:time
prov:type
wasAttributedTo
Software
type= prov:SoftwareAgent
prov:label
wasAssociatedWith
prov:startTime
prov:endTime
prov:type

Standard Graph Visualizations and Textual Representations of
Provenance Data are not Easy to Understand by Non-experts

Idea: Provenance Visualization Using Comics
Provenance Comics
• Presenting the provenance of processes in visual representation that people can understand
without prior instructions or training (“Provenance for people”)
• Assumption
• People are familiar with comics from every day life
• See daily strips in newspapers etc.

Provenance Comics
Design considerations
• Data provenance has a temporal aspect: origin, manipulation, transformation,
and other activities happen sequentially over time
• The directed acyclic provenance graph guarantees that, while moving
through its nodes, one always moves linearly forward in time
• It’s possible to derive a temporal sequence of happenings from
the graph that can be narrated like a story
Mapping provenance graph to comics
• We generate a comic strip for each basic activity in the provenance graph
• Each strip consists of a varying number of panels, which are small drawings
that provide further details about the activity
• The complete set of comic strips shows the “story” of the data

First Sketches

Current Graphical Style

Single Comic Strip Shows a Single Data-related Action

Communicate to People Where Data is Stored

Understand How Data is Analyzed

Distinctive Features
• Shapes
• Colors
• Icons
• Letters
• Labels

Representation of PROV Elements
Agents
Entities
Activity-related

Collecting QS Provenance
Weight Tracking App
https://play.google.com/store/apps/details?id=de.medando.weightcompanion

Collecting QS Provenance
Visualization with Python Script

Date,Time,Weight,Waist,Hip,Device,Comment
"Jun 13, 2012",14:00,83.7,,,Withings,
"Jun 13, 2012",14:08,79.7,,,Withings,
"Jun 15, 2012",21:59,82.7,,,Withings,
"Jun 15, 2012",22:04,82.7,,,Withings,
"Jun 24, 2012",18:32,86.1,,,Withings,
"Jun 26, 2012",07:42,80.8,,,Withings,
"Jun 27, 2012",07:40,81.1,,,Withings,
"Jun 29, 2012",07:34,79.4,,,Withings,
"Jun 30, 2012",22:12,81.7,,,Withings,
"Jul 1, 2012",11:21,80.6,,,Withings,
"Jul 7, 2012",17:04,80.7,,,Withings,
"Jul 10, 2012",07:46,81.8,,,Withings,
"Jul 11, 2012",07:32,78.6,,,Withings,
"Jul 12, 2012",07:26,79.4,,,Withings,

PROV Comics
Web Application
http://provcomics.de
• Implemented in JavaScript
• Single page website
• Reads provenance graph from
PROVSTORE
• Uses PROVSTORE jQuery API
• Code:
http://github.com/DLR-SC/prov-
comics

Implementation Details
Additional attributes
agent(qs:app/stepcounter, [prov:type="prov:SoftwareAgent",
qs:device="smartphone", prov:label="StepCounter"])
agent(qs:service/fitbit,
[prov:type="prov:Organization", prov:label="Fitbit"])
wasGeneratedBy(userdata:activities/steps, method:request,
2016-12-01T16:06:22+00:00, [prov:role="uploading"])
http://provcomics.de/?username=rstruminski&docId=115547

Open Issues
Current implementation is a prototype with limitations
• Flexibility and generalization
• Handling of
• large provenance graphs
• incomplete provenance data
• branches and multiple data sources
• Expects a single PROV document

Future Work and Use Cases
Future Work Possible Use Cases
• Different comic styles
• Comparative user studies
• Quantitative comics
• Geographical information
• Glyph-based depiction
• Technical improvements
• Large Provenance graphs
• Provenance templates
• “Intelligent” generation of pictures
• Journalism
• Generation of handbooks
• Communicating incidents

Thank You!
Andreas Schreiber
www.DLR.de/sc/ivs
andreas.schreiber@dlr.de
@onyame

Visualizing Provenance using Comics

More Related Content

More from Andreas Schreiber (20)

Recently uploaded (20)

Visualizing Provenance using Comics