No Master Algorithm: Human-machine intelligence and the real-world needs of fact-checking and content moderation
0 likes17 views
The document discusses the development of automated fact-checking systems that balance academic and practitioner goals, focusing on identifying and managing claims needing verification. It emphasizes the need for collaboration in data collection, analysis, and establishing best practices for data governance amidst closed networks. Furthermore, it highlights the importance of local knowledge in content moderation and outlines various research partnerships and methodologies for improving fact-checking processes.
No Master Algorithm: Human-machine intelligence and the real-world needs of fact-checking and content moderation
- 1. No master algorithm Human-machine intelligence and the real-world needs of fact-checking and content moderation Dr Scott A. Hale, @computermacgyve
- 2. Toward Automated Fact-Checking: ...Detecting Check-worthy Factual Claims… ...Developing an Annotation Schema...for Claim Detection
- 3. Academic vs. practitioner goals Academic: Build a system to identify claims needing to be fact-checked / moderated Build a system to do fact-checking / moderation
- 4. Academic vs. practitioner goals Academic: Build a system to identify claims needing to be fact-checked / moderated Build a system to do fact-checking / moderation Practitioner: Wants to select claims to fact-check based on organisational context, own knowledge/expertise, etc. Needs assistance prioritising claims (based on reach, impact, domain, type) Needs assistance managing pool of experts, workload, etc.
- 5. We need to work together Problem motivation Data collection, analysis Method development Extracting and clustering claims Matching similar images
- 7. Building a data trust to ensure data is open for research Partnerships Harvard CyberLaw Clinic -Kendra Albert Wolfe and Hwang -Tim Hwang Digital Public -Sean McDonald Conduct in-depth research focused on usability, scalability, and security Establish best practices for a new type of data governance in the era of closed networks. Image source: Open Data Institute
- 8. Claim extraction and clustering Konstantinovskiy et al. (2018) arXiv:1809.08193
- 9. Image matching PDQ Perceptual hashing Neural network representations pHash distance 6
- 10. pHash distance 4
- 11. pHash distance 4
- 12. Dialogue Credibility Coalition (CredCo): https://credibilitycoalition.org/ Working groups on UX, InfoSec, Images & Memes, Health, Literacy,.... Content Moderation Group (Como)
- 16. Original research and research partnerships Image and claim matching UX Health fact-checking Importance of local knowledge in content moderation
- 17. Thank You Dr Scott Hale @computermacgyve