Poio API and GraF-XML @ Balisage 2013
Download as ODP, PDF0 likes556 views
This document discusses standards and tools for language documentation and annotation, including existing infrastructure, CLARIN, annotation graphs, the Graph Annotation Framework (GrAF) and Poio API. It provides an example of mapping annotations from an ELAN EAF file to GrAF-XML using the Poio API and discusses analysis workflows and the CLASS project.
![Annotation Graphs
● the underlying data model for linguistic annotations
● pivot structure for linguistic data
● time vs. byte offsets
● not hierarchical (but trees are also graphs)
● stand-off annotation
● "It is important to recognize that translation into AGs does
not magically create compatibility among systems whose
semantics are different." [Bird & Liberman 2001]](https://image.slidesharecdn.com/balisage2013-130821054413-phpapp01/85/Poio-API-and-GraF-XML-Balisage-2013-7-320.jpg)
![Elan EAF
<TIER DEFAULT_LOCALE="en" LINGUISTIC_TYPE_REF="words"
PARENT_REF="W-Spch" PARTICIPANT="" TIER_ID="W-Words">
<ANNOTATION>
<ALIGNABLE_ANNOTATION ANNOTATION_ID="a23"
TIME_SLOT_REF1="ts4" TIME_SLOT_REF2="ts6">
<ANNOTATION_VALUE>so</ANNOTATION_VALUE>
</ALIGNABLE_ANNOTATION>
</ANNOTATION>
<ANNOTATION>
[...]
</ANNOTATION>
</TIER>](https://image.slidesharecdn.com/balisage2013-130821054413-phpapp01/85/Poio-API-and-GraF-XML-Balisage-2013-14-320.jpg)
![Tier hierarchies
[
['utterance..K-Spch'],
['utterance..W-Spch',
['words..W-Words',
['part_of_speech..W-POS']
],
['phonetic_transcription..W-IPA']
],
['gestures..W-RGU',
['gesture_phases..W-RGph',
['gesture_meaning..W-RGMe']
]
],
['gestures..K-RGU',
['gesture_phases..K-RGph',
['gesture_meaning..K-RGMe']
]
]
]](https://image.slidesharecdn.com/balisage2013-130821054413-phpapp01/85/Poio-API-and-GraF-XML-Balisage-2013-18-320.jpg)
Poio API and GraF-XML @ Balisage 2013
- 1. Poio API and GrAF-XML A radical stand-off approach in language documentation and language typology Jonathan Blumtritt, Cologne Center for eHumanities, University of Cologne Peter Bouda, Centro Interdisciplinar de Documentação Linguística e Social Felix Rau, Department of Linguistics, University of Cologne
- 2. Overview ● Existing infrastructure and workflows ● CLARIN ● Annotation graphs ● GrAF and Poio API ● Example: Elan EAF to GrAF-XML ● CLASS
- 5. LD tools and standards ● Elan: EAF, MPEG, WAV ● Toolbox: TXT, XML, WAV ● Arbil: IMDI/CIMDI („Component MetaData Infrastructure“) ● Praat: XML, WAV ● ... ● No standards for tier hierarchies, tier names or annotation schemes ● Efforts in ISOcat
- 6. ● European initiative within the European Research Infrastructure Consortium: Common Language Resources and Technology Infrastructure (CLARIN) ● aims at providing easy and sustainable access for scholars in the humanities and social sciences to digital language data ● Started in 2006, part of a roadmap process, timeline currently ending 2020 ● CLARIN-D: working groups in Germany ● Curation projects for different research areas in linguistics
- 7. Annotation Graphs ● the underlying data model for linguistic annotations ● pivot structure for linguistic data ● time vs. byte offsets ● not hierarchical (but trees are also graphs) ● stand-off annotation ● "It is important to recognize that translation into AGs does not magically create compatibility among systems whose semantics are different." [Bird & Liberman 2001]
- 9. GrAF ● GrAF: Graph Annotation Framework ● ISO 24612: Language resource management - Linguistic annotation framework (LAF) ● Started as stand-off version of XCES ● API and representation as data structures, not a file format ● GrAF/XML as XML representation ● Used for the MASC of the ANC ● Nodes, edges, regions, annotations, feature structures
- 10. TEI and GrAF ● Schemata for GrAF created with TEI Roma ● Custumized version of TEI P5 schema ● ODD: „One Document Does it all“ ● GrAF is not TEI compliant ● Share data types and feature structures of annotations ● TEI has „stand-off“ variant, uses XPointer/XLink – Primary data has to be XML
- 11. Why we use GrAF ● Because it's new! :-) ● No inline markup ● Radical stand-off approach – Easier to share and manage data – Preferred solution to archive cultural heritage – Ideal for sparse annotations ● Existing code: Java and Python ● The beauty of annotation graphs
- 12. Poio API ● Think of GrAF as an assembly language for linguistic annotation; then Poio API is a libray to map from and to higher-level languages ● Subset of GrAF to represent tier based annotation ● Filters and filter chains for search ● Plugin mechanism for file formats – Mapping semantics: tiers and annotations to nodes and edges ● Meta-data for additional information (tier types etc.)
- 13. Example: Mapping of EAF to GrAF-XML
- 14. Elan EAF <TIER DEFAULT_LOCALE="en" LINGUISTIC_TYPE_REF="words" PARENT_REF="W-Spch" PARTICIPANT="" TIER_ID="W-Words"> <ANNOTATION> <ALIGNABLE_ANNOTATION ANNOTATION_ID="a23" TIME_SLOT_REF1="ts4" TIME_SLOT_REF2="ts6"> <ANNOTATION_VALUE>so</ANNOTATION_VALUE> </ALIGNABLE_ANNOTATION> </ANNOTATION> <ANNOTATION> [...] </ANNOTATION> </TIER>
- 15. GrAF entities
- 16. GrAF structure
- 17. GrAF-XML <node xml:id="words..W-Words..na23"> <link targets="words..W-Words..ra23"/> </node> <region anchors="780 1340" xml:id="words..W-Words..ra23"/> <edge from="utterance..W-Spch..n8" to="words..W-Words..na23" xml:id="ea23"/> <a as="words" label="words" ref="words..W-Words..na23" xml:id="a23"> <fs> <f name="annotation_value">so</f> </fs> </a>
- 19. The code ag = poioapi.annotationgraph.AnnotationGraph() parser = poioapi.io.ElanParser("example.eaf") writer = poioapi.io.graf.Writer() converter = poioapi.io.graf.GrAFConverter(parser, writer) converter.parse() converter.write("example.hdr")
- 20. Analysis workflows ● Graph-based methods ● Pipe to scientific Python libraries ● GrAF connectors for major linguistic workflow tools (GATE and Apache UIMA) ● Example: Polysemy in dictionaries ● Example: Counting word orders
- 21. CLASS
- 22. Thank you for your attention! pbouda@cidles.eu
- 23. Links Clarin curation project: http://de.clarin.eu/en/discipline-specific-working-groups/wg-3-linguistic-fieldwork-anthr Poio API: http://media.cidles.eu/poio/poio-api/ GrAF: http://www.xces.org/ns/GrAF/1.0/ CLASS: http://class.uni-koeln.de