Fighting Misinformation in Cyberspace
Download as PPTX, PDF0 likes44 views
The document discusses challenges around misinformation, hate speech, and violent extremism online. It outlines some potential solutions companies could take, including defining clear policies, changing products/algorithms, and manual review processes. Specifically, it recommends building better signals to predict quality content, improving detection of queries requiring quality results, and implementing "flight to quality" algorithms to boost trusted sources for at-risk topics. It also discusses proactively monitoring for misinformation during crises and extending reactive processes used in crises to broader monitoring.
![Solutions are Addressing the Root Causes
Problem Prediction System Response
Ranking &
Intelligence
Policy Narrative
Fringe Query
Expansion
Impacts:
Misinformation, violent
extremism
Keynotes:
Algorithmically
increase coverage of
breaking news and
violent extremism
queries for flight to
quality launches.
VE Content
Ranking | Eval
Impacts: Violent
extremism
Keynotes: Leverage
deep experts to
inform
inciting/recruiting VE
content. Brings core
company values into
ranking.
Manual
Operations
Impacts:
Misinformation,
violent extremism
Keynotes: Manual
work (process) to
product stop-gap
solutions, e.g.
engaging flight to
quality for at risk
topics during crises
Foreign Actor
Ops Penalties
Impacts:
Misinformation,
election interference
Keynotes: Threat
analysis of
misinformation ops.
Penalties include
removal from news
surfaces [and spam
penalties elsewhere]
Early Alert
System
Impacts:
Misinformation,
violent extremism,
election
interference, hate
Keynotes: Efforts to
start systematic
adversarial testing
company products
Narrative
Development
Impacts: Violent
extremism, hate,
misinformation
Keynotes: Policy,
Search and PR are
coordinating to
create core narratives
around corporate
values [and product
promises]](https://image.slidesharecdn.com/misinformation-190925201446/85/Fighting-Misinformation-in-Cyberspace-13-320.jpg)
Fighting Misinformation in Cyberspace
- 1. Confidential + Proprietary Fighting Misinformation and Violent Extremism/Hate Content in CyberSpace
- 2. Recent News Cycles have Highlighted a Set of Challenges: ● Violent/Extremism Content ● Hate Speech ● Misinformation/Fake News ● (Cross border) Disinformation Companies have to develop new solutions or strengthen existing solutions.
- 3. Search has a delicate balancing act in accomplishing our mission to get what a user wants while respecting our ethics as a company Some Related Challenges Violent Extremism / Hate Speech Cross-Border Disinformation Other Disinformation / Fake News* Violent Extremism: Content that recruits, fundraises, or promotes violence or terrorism. Examples include content meant to incite violence, celebrate terrorist attacks, or promote acts of terrorism. Graphic content in an educational (such as news), documentary, scientific, or artistic context, should be indicated as such. Hate Speech: While preserving freedom of expression, content that threatens or advocates for harm on oneself or others; harasses, intimidates or bullies an individual or group of individuals; incites hatred against, promotes discrimination of, or disparages an individual or group on the basis of their race or ethnic origin, religion, disability, age, nationality, veteran status, sexual orientation, gender, gender identity, or other characteristic that is associated with systemic discrimination or marginalization Coordinated Inauthentic Activity: Cross- border coordinated activity. Generally driven by political motives, rather than profit motives. Examples: false popularity/virality produced via bots or troll farms. State-sponsored Propaganda: Content that is placed by governments intended to change or sway public perception Imposter or Misleading or Deceptive “News”: Satire portrayed as truth, abc.com.co, 70news Clickbait or Sensationalist or Tabloidy content: “10 Shocking Things”, “Oprah’s secret revealed!” Fringy or Conspiracy or Controversial Opinion: Holocaust denial, 9/11 hoax, Vaccines cause autism *Fake News analysis to span all products, not just News products
- 4. Three Types of Solutions to Avoid These Problems for Companies Define policies and standards to make decisions about products Policy Change products to improve results or implement policies, e.g. changing algorithms Product Manual (tool assisted) work to collect information, produce analyses and insights, implement manual steps Operations
- 5. Misinformation and Violent Extremism/Hate are Different Misinformation Violent/Extremism
- 6. Two Integrated Workstreams to Address These Problems Systemic Solutions ● Building better signals with improved notions of content quality and better models to predict quality ● Improving detection of queries where the focus should be on the quality of results (0.2%-0.5% of traffic). ● Implementing flight to quality algorithms that change the weight of some signals and/or turn off certain features ○ General fringe classifier ○ Incident fringe classifier ○ Violent Extremism (V/E) classifier Policy Solutions ● Reduced reactive cycle to <48 hours. ● Improved process for approvals of takedowns ● Implemented emergency processes to quickly react to misinformation in Crisis contexts in selected countries ● Extending this work to cover reactive monitoring, V/E content, global scope
- 7. Confidential + Proprietary Policy Solutions
- 8. Privileged + Confidential Policy & Enforcement Current State: Violent Extremism / Hate Speech Ethos Targeted Policy? Enforcement Options Details Open Web: Preserve freedom of information while tolerating a few content restrictions to promote quality and user safety. Prefer algorithmic over manual solutions. No Legal Removals + Quality Approach Better Ranking VE & Hate: Exploring engaging VE external experts to support better classification of harmful VE content Yes Removal VE: Strictly prohibit content related to terrorism. We also do not permit terrorist organizations to use product. Examples include but are not limited to: Using product to incite violence, celebrate terrorist attacks, promote acts of terrorism. If posting graphic content in an educational (such as news), documentary, scientific, or artistic context, please be mindful to provide enough information to help people understand what’s going on. Hate: Content that promotes discrimination or hatred, or that incites violence against individuals or groups based on based on race, ethnic origin, nationality, sex, gender identity, sexual orientation, religion, disability, age, veteran status, or any other characteristic that is associated with systemic discrimination or marginalization is prohibited. Curated Data: Emphasize quality, accuracy, ideological balance, and empathy for users. Yes Removal Users & News Publishers: Promote freedom of expression while preserving on-topic, civil discourse Yes Removal 8
- 9. Privileged + Confidential Ethos Targeted Policy? Enforcement Options Details Open Web: Preserve freedom of information while tolerating few content restrictions to promote quality and user safety. Prefer algorithmic solutions. No Better Ranking Whether Search can derive a quality signal or spam policy leveraging the below Cross-Border Policy.* Current: Penalty in News Corpus will impact Search Ranking. Under Develop. Demotion or Removal Cross-border Disinformation Conduct Policy: Targets behavioral factors, i.e., networked activity between seemingly independent sites, unnatural traffic patterns, geo mismatch of domain with target audience of site and concealing connections to a state entity while targeting foreign audiences. Curated Data: Emphasize quality, accuracy, ideological balance, and empathy for users. Users & News Publishers: Promote freedom of expression while preserving on-topic, civil discourse 9 Policy & Enforcement: Cross-border Disinformation
- 10. Privileged + Confidential Policy & Enforcement: Other Disinformation Ethos Targeted Policy? Enforcement Options Details Open Web: Preserve freedom of information while tolerating few content restrictions to promote quality and user safety. Prefer algorithmic solutions. No Better Ranking Rater guidelines for lower scoring of demonstrably inaccurate content Increasing quality for breaking news, violent incident queries where inaccurate info is large risk Partial Removal Blacklisting WebAnswers that are egregiously or offensively inaccurate. Curated Data: Emphasize quality, accuracy, ideological balance, and empathy for users. Yes (various) Removal Using structured data reviewed by experts and using web evidence to resolve factual disputes in structured data Users & News Publishers: Promote freedom of expression while preserving on-topic, civil discourse Yes Removal News publishers may not misrepresent, misstate, or conceal information about the site’s owner or primary purpose. News sites may not misrepresent identity or content. Removal from corpus if publisher repeatedly disproven by fact checkers. 10
- 11. Confidential + Proprietary Systemic Solutions
- 12. Different Root Causes at Play Are companies adversarially testing themselves? Problem Prediction Correct balancing of algo and manual response? System Response Are companies identifying “good” vs.“bad” content? Bad actors? Are companies sufficiently explaining our values and promises? Ranking & Intelligence Policy Narrative
- 13. Solutions are Addressing the Root Causes Problem Prediction System Response Ranking & Intelligence Policy Narrative Fringe Query Expansion Impacts: Misinformation, violent extremism Keynotes: Algorithmically increase coverage of breaking news and violent extremism queries for flight to quality launches. VE Content Ranking | Eval Impacts: Violent extremism Keynotes: Leverage deep experts to inform inciting/recruiting VE content. Brings core company values into ranking. Manual Operations Impacts: Misinformation, violent extremism Keynotes: Manual work (process) to product stop-gap solutions, e.g. engaging flight to quality for at risk topics during crises Foreign Actor Ops Penalties Impacts: Misinformation, election interference Keynotes: Threat analysis of misinformation ops. Penalties include removal from news surfaces [and spam penalties elsewhere] Early Alert System Impacts: Misinformation, violent extremism, election interference, hate Keynotes: Efforts to start systematic adversarial testing company products Narrative Development Impacts: Violent extremism, hate, misinformation Keynotes: Policy, Search and PR are coordinating to create core narratives around corporate values [and product promises]
- 14. Some Steps Recommended to be Taken Algorithmic Changes ● Built better signals with improved notions of content quality and better models to predict quality ● Improved detection of queries where companies need to stress quality of results (0.2%-0.5% of traffic). ● Implemented flight to quality algorithms that change the weight of some signals and/or turn off certain features ○ General fringe classifier ○ Incident fringe classifier ○ Violent Extremism classifier
- 15. Reactive Monitoring of Misinformation Content in Crises - In crisis situations, some search engines temporarily show poor quality results for some queries because there are few high quality ones. There are company teams that already stand up in crisis situations. Companies are already leveraging their capabilities to help create a temporary, manual process solution until algorithmic ones are developed.
- 16. Process Example for Reactive Monitoring During Crisis Policy Define and approve a clear policy for identifying which queries should trigger flight to safety. Stand Down Stand down the flight to safety after higher quality content is available Identify/Monitor Extend existing Crisis Response/SOS and T&S processes to include the identification and monitoring of a set of queries that require a flight for safety in Crisis situations. Escalate/Enact Extend existing T&S processes to escalate these queries to the correct escalation path and trigger flight to safety
- 17. Extend to Proactive Monitoring, Expand Scope to Misinformation + Violent Extremism + Hate Content Build operations team to run monitoring 24/7 Define play-books for 90% of the escalations, including PR proactive positioning on top areas Establish types of queries that always trigger flight to quality Develop feedback loop to help development of automated solutions Broaden scope from selected EN markets to most key global markets Build tools to support manual monitoring
- 18. Search has a delicate balancing act in accomplishing our mission to get what a user wants while respecting our ethics as a company Different but Connected Challenges in Misinformation Violent Extremism / Hate Speech Cross-Border Disinformation Other Disinformation / Fake News* Violent Extremism: Content that recruits, fundraises, or promotes violence or terrorism. Examples include content meant to incite violence, celebrate terrorist attacks, or promote acts of terrorism. Graphic content in an educational (such as news), documentary, scientific, or artistic context, should be indicated as such. Hate Speech: While preserving freedom of expression, content that threatens or advocates for harm on oneself or others; harasses, intimidates or bullies an individual or group of individuals; incites hatred against, promotes discrimination of, or disparages an individual or group on the basis of their race or ethnic origin, religion, disability, age, nationality, veteran status, sexual orientation, gender, gender identity, or other characteristic that is associated with systemic discrimination or marginalization Coordinated inauthentic activity: Cross- border coordinated activity. Generally driven by political motives, rather than profit motives. Examples: false popularity/virality produced via bots or troll farms. State-sponsored Propaganda: Content that is placed by governments intended to change or sway public perception Imposter or misleading or deceptive “news”: Satire portrayed as truth, abc.com.co, 70news Clickbait or sensationalist or tabloidy content: “10 shocking things”, “Oprah’s secret revealed!” Fringy or conspiracy or controversial opinion: Holocaust denial, 9/11 hoax, vaccines cause autism *Fake News analysis to span all products, not just News products
- 19. Several Components to Flight to Quality Changes 1. Build better signals a. Improved notions of page quality (updated rater guidelines to make clear quality includes non-structural features such that news has to be true - pages or sites need to be credible). b. Changed our models to use the raters data to improve prediction of quality signals 2. Detect queries where it is more important than average to stress quality of results - between 0.2%-0.5% of traffic. 3. In case such queries are detected, the technical flight to quality starts by changing the weight of some signals - e.g. increase weight of quality, reduce weight of clicks, cap topicality. a. Building fringe classifier b. Building specific models for incident fringe classifier with limited flight to quality c. Building Violent Extremism classifier - suppressing certain features (e.g. videos) for mostly news queries, giving higher weight to both authoritative and fresh content
- 20. Policy and Applicable Search Features Misinformation Query patterns that could lead to potentially harmful misinformation. In a non-crisis situation monitoring relies on a calendar of events, breaking news and dictionary of queries. Flight to safety News Feed Violent extremism Content that recruits, fundraises, or promotes violence or terrorism. Examples include content meant to incite violence, celebrate terrorist attacks, or promote acts of terrorism. Graphic content in an educational (such as news), documentary, scientific, or artistic context, should be indicated as such. Webanswers Suggest Knowledge Engine Structured Data Highlights News Feed Image (Safe Search) Hate Content that promotes, glorifies, or condones violence or has the primary purpose of Inciting Hatred against a protected group or a member of a protected group based on the protected characteristic (race, ethnic origin, religion, disability, age, nationality, veteran status, sexual orientation, gender, gender identity).
Editor's Notes
- #6: Specific steps we have taken include: 1. Algorithm changes like demotion for XX, YY, ZZ - incident fringe 2. Reducing reactive cycle to <48 hours. -- Process for approvals -- Identifying bad queries (using Crisis response process) 3. Improved signals
- #9: *Impacts ranking/q-score
- #10: *Discussions starting with webspam
- #14: Specific steps to be taken include: 1. Algorithm changes like demotion for XX, YY, ZZ - incident fringe 2. Reducing reactive cycle to <48 hours. -- Process for approvals -- Identifying bad queries (using Crisis response process) 3. Improved signals
- #15: Specific steps we have taken include: 1. Algorithm changes like demotion for XX, YY, ZZ - incident fringe 2. Reducing our reactive cycle to <48 hours. -- Process for approvals -- Identifying bad queries (using Crisis response process) 3. Improved signals