Annotation examples--Fribourg--2019-09-03
- 1. Annotation Jodi Schneider Linguistic and corpus perspectives on argumentative discourse, SwissUniversities Doctoral Programme Language & Cognition University of Fribourg, Fribourg, Switzerland 2019-09-02
- 2. A typical annotation process • Find text of interest • Find phenomena of interest • Draft an annotation manual • Iteratively test annotation & revise manual – Find questionable annotations, check disagreements. – Revise the manual. – Iterate. • Annotate
- 3. Examples of annotation software • GATE: https://gate.ac.uk Free & Open source – NLP pipeline integration, robust developmnet community, ingests lots of formats, • UAM CorpusTool: http://www.corpustool.com Free – Comparative statistics, corpus search, annotation schemes easy to set up • Excel – Great for simple annotation • BRAT: http://brat.nlplab.org Free & Open source – Run your own instance, browser-based for collaboration • EPPI Reviewer: https://eppi.ioe.ac.uk/cms/Default.aspx?alias=eppi.ioe.ac.uk/cms/er4 – Data extraction for systematic review • Custom tools
- 4. GATE Jodi Schneider, Alexandre Passant, and Stefan Decker “Deletion Discussions in Wikipedia: Decision Factors and Outcomes.” In WikiSym2012. Linz, Austria, August 27-29, 2012.
- 5. UAM CorpusTool(V 2.8.16) Jodi Schneider, Krystian Samp, Alexandre Passant, Stefan Decker. “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad- hoc Online Task Groups”. In CSCW 2013, San Antonio, TX, February 23-27, 2013.
- 6. Excel Dong, Xiaoru; Xie, Jingyi; Hoang, Linh (2019): Inclusion_Criteria_Annotation. University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB-5958960_V2 for Text Mining Pipeline to Accelerate Systematic Reviews in Evidence-Based Medicine
- 7. Excel Hoang, Linh; Schneider, Jodi (2018): Citation context analysis of RobotReviewer core papers circa 2018-06. University of Illinois at Urbana-Champaign. https://doi.org/10.13012/B2IDB- 1075526_V1 for Text Mining Pipeline to Accelerate Systematic Reviews in Evidence-Based Medicine
- 8. BRAT Halil Kilicoglu, Zeshan Peng Shabnam Tafreshi, Tung Tran, Graciela Rosemblat, Jodi Schneider. “Confirm or Refute?: A Comparative Study on Citation Sentiment Classification in Clinical Research Publications.” Journal of Biomedical Informatics, Vol 91, 103123. doi: 10.1016/j.jbi.2019.103123
- 9. EPPI-Reviewer Work in progress, Systematic Review of Empirical Research about Retracted Publications project team
- 10. Custom Tools Halil Kilicoglu, Graciela Rosemblat, Zeshan Peng, Mario Malicki, Tony Tse, Jodi Schneider, Gerben ter Riet. Annotating Clinical Trial Publications to Assess CONSORT Adherence: A Feasibility Study. 6th World Conference on Research Integrity, Hong Kong, 2019.
- 12. Some tools have additional features
- 13. GATE - sentiment (gazeteer) Jodi Schneider & Adam Wyner. “Identifying Consumers' Arguments in Text” In SWAIE 2012: Semantic Web and Information Extraction at EKAW 2012.
- 14. GATE - , , (gazeteers) Jodi Schneider & Adam Wyner. “Identifying Consumers' Arguments in Text” In SWAIE 2012: Semantic Web and Information Extraction at EKAW 2012.
- 15. GATE – semantic search Jodi Schneider & Adam Wyner. “Identifying Consumers' Arguments in Text” In SWAIE 2012: Semantic Web and Information Extraction at EKAW 2012.
- 16. UAM CorpusTool(V 2.8.16) Jodi Schneider, Krystian Samp, Alexandre Passant, Stefan Decker. “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad- hoc Online Task Groups”. In CSCW 2013, San Antonio, TX, February 23-27, 2013.
- 17. UAM CorpusTool(V 2.8.16) Jodi Schneider, Krystian Samp, Alexandre Passant, Stefan Decker. “Arguments about Deletion: How Experience Improves the Acceptability of Arguments in Ad- hoc Online Task Groups”. In CSCW 2013, San Antonio, TX, February 23-27, 2013.
- 18. EPPI-Reviewer Work in progress, Systematic Review of Empirical Research about Retracted Publications project team