The Death of the Keyword: In Search of NLP -- Presented at Ungagged London Apr 2019.

#GetUngagged @schachin
Kristine Schachinger
The Death of the Keyword: In Search of NLP
Kristine@SitesWithoutWalls.com

In the beginning, there was a …
Large-Scale Hypertextual Web Search Engine

What?

Link Profiles
http://infolab.stanford.edu/pub/papers/google.pdf

The Web 1998

Google Goes To Work
http://infolab.stanford.edu/pub/papers/google.pdf

Today in 2019 …
Roughly half of the world's population or
3.8 billion people use the internet every day.

Google
processes
TRILLIONS
of queries a
year & has
indexed
BILLIONS
of Websites.

Yesterday, THERE WERE OVER
438,662,584 Google searches.http://www.internetlivestats.com/google-search-statistics/

Dealing With The Data.

Google Search
was founded on
unstructured
data.

Unstructured data (or unstructured information) is information that
either does not have a pre-defined data model or is not organized in a
pre-defined manner. Unstructured information is typically text-heavy,
but may contain data such as dates, numbers, and facts as well.
https://www.google.co.uk/search?q=definition+unstructured+data&oq=definition+unstructured+data&aqs=chrome..69i57j0l5.5175j0j7&sourceid=chrome&ie=UTF-8

Unstructured data (or unstructured information) is information that
either does not have a pre-defined data model or is not organized in a
pre-defined manner. Unstructured information is typically text-heavy,
but may contain data such as dates, numbers, and facts as well.
https://www.google.co.uk/search?q=definition+unstructured+data&oq=definition+unstructured+data&aqs=chrome..69i57j0l5.5175j0j7&sourceid=chrome&ie=UTF-8
This is known as the “Bag of Words” approach.

Otherwise Known as Keywords.

Unstructured Data
uses keywords.
https://moz.com/blog/7-advanced-seo-concepts

TF-IDF
Term Frequency Inverse
Document Frequency
ie the frequency of keywords
compared to other
documents in the set

But as queries number in the trillions
unstructured data becomes inefficient.
Data needed structure.

https://twitter.com/manishrjain
https://blog.dgraph.io/post/why-google-needed-graph-serving-system/

https://blog.dgraph.io/post/why-google-needed-graph-serving-system/

Semantics: Keywords to Queries.

So Google moved from
Relational Databases to
Knowledge Graphs.
Knowledge Graphs

NOTE
Knowledge Graphs
DO NOT EQUAL
THE KNOWLEDGE GRAPH
Knowledge Graphs

“Graph-based knowledge representation has been
researched for decades and the term knowledge
graph does not constitute a new technology.
Rather, it is a buzzword reinvented by Google
and adopted by other companies and academia to
describe different knowledge representation
applications.”
Knowledge Graphs
http://ceur-ws.org/Vol-1695/paper4.pdf

Enter Semantic Search
https://web.archive.org/web/20090516213508/http://blog.searchenginewatch.com/090512-201139

What is Semantic Search?

Semantic Search
=
Understanding Intent

Why?

The Holy Grail
of Search?
NLP
(Natural Language
Processing)

Welcome THE Knowledge Graph 2012.

Knowledge Graphs are based on known relationships.
THE Knowledge Graph is Google’s graph database.
THE Knowledge Graph

https://searchengineland.com/up-close-google-squared-19313
Before the Knowledge Graph – Wonder Wheel

(Knowledge Graphs)
”…quite possibly ...
one of Google's significant achievements”
Nathania Johnson of Search Engine Watch
Knowledge Graphs

The Knowledge Graph (Google) is seeded by things known.
Instead of just text without meaning, The KG is a relational
graph with known objects and mapped relationships.
THE Knowledge Graph

Why the Knowledge Graph?
To help better match user intent.
To understand what users want.
THE Knowledge Graph

Why?
Google doesn’t truly process
Natural Language (NLP), but it does use
Natural Language Understanding (NLU)
The Knowledge Graph was the first step
towards language understanding.

NLU is a
subset
of NLP.

Enter Machine Learning

https://www.tensorflow.org/guide/
MACHINE LEARNING

The Knowledge Graph enables you to search for things, people or places
that Google knows about—landmarks, celebrities, cities, sports teams,
buildings, geographical features, movies, celestial objects, works of art
and more—and instantly get information that’s relevant to your query
THE Knowledge Graph

In other words
NOUNS
THE Knowledge Graph

NOUNS=ENTITIES
THE Knowledge Graph

Google moves to
ENTITY SEARCH
THE Knowledge Graph

Knowledge Graph entities
The Knowledge Graph has millions of entries that describe real-world entities like people, places, and things. These
entities form the nodes of the graph.
The following are some of the types of entities found in the Knowledge Graph:
Book
BookSeries
EducationalOrganization
Event
GovernmentOrganization
LocalBusiness
Movie
MovieSeries
MusicAlbum
MusicGroup
MusicRecording
Organization
Periodical
Person
Place
SportsTeam
TVEpisode
TVSeries
VideoGame
VideoGameSeries
WebSite
THE Knowledge Graph ENTITIES

Entities (nouns) + Relationships (nodes)
=
THE Knowledge Graph
THE Knowledge Graph

Knowledge Graph
= the Answer Engine
THE Knowledge Graph

Google as an Answer Engine
https://www.google.com/search/howsearchworks/responses/#?modal_active=none

THE Knowledge Graph Seeds.

"Four years ago this July, Google
acquired Metaweb,
bringing Freebase and
linked open data to Google,"
he wrote.
Google software engineer Barak Michener
http://www.eweek.com/database/google-releases-cayley-open-source-graph-database
THE Knowledge Graph Seeds

Also includes trusted
sources such as the
CIA Fact Book, Wikipedia,
Wikidata etc.
http://www.eweek.com/database/google-releases-cayley-open-source-graph-database
THE Knowledge Graph Seeds

Hummingbird 
“Strings to Things”.
The name was derived from
the speed and accuracy of the hummingbird.

Hummingbird Arrives 2013
Google moves from matching keyword terms to
Google trying to process Natural Language Queries
using the Knowledge Graph.
“Strings to Things"

But Google doesn’t process
Natural Language very well.

Hummingbird was the first step in Google moving
search from strings (unstructured data)
or the “bag of words” approach
 to “things” (structured data)

“Things” are known objects
with known (or learned) relationships.

Things Need Context.
Moving to Vector Relationships.

KEY FACTOR word2vec:
Vector space models (VSMs) represent (embed)
words in a continuous vector space where
semantically similar words are mapped to nearby
points ('are embedded nearby each other').
Hummingbird
https://www.tensorflow.org/tutorials/representation/word2vec

Embedded Word Model
Hummingbird

“…words that appear in the same contexts share semantic meaning. The
different approaches that leverage this principle can be divided into two
categories: count-based methods (e.g. Latent Semantic Analysis),
and predictive methods (e.g. neural probabilistic language models).”
Hummingbird

Hummingbird

Hummingbird added a semantic layer
to the search algorithms.

Not Just Keywords Anymore …
Semantic Interpretations.

Hummingbird adds a
semantic layer to the
search algorithms like
synonyms and close
variants.

Hummingbird adds a
search algorithms that
uses “semantic distance
and term relationships”.

Hummingbird adds a
search algorithms that
uses “phrase based
Indexing and co-
occurrence.”

Page Segmentation.
This part of the
algorithm determines
meaning through
placement.

Entity Salience.
This part of the
algorithm determines
meaning through known
relationships.

Entity Salience.
This part of the algorithm
determines meaning
through known
relationships.
+
2018-19 Google adds the
“topic layer” to the
knowledge graph
(categorical classification)

So Hummingbird moves from
strict word count based modeling
(ie keyword counts) to
probabilistic modeling
(ie predictive interpretation)
via known word vectors+nodes (relationships).
Hummingbird

What does this look like mathematically?

BUT …..
Google Search still
doesn’t process
Natural Language in
the true sense.

This means we must add an “interpreter”.

Enter Structured Data and Schema.

What is Structured Data?
https://developers.google.com/search/docs/guides/intro-structured-data

Structured data for SEO purposes is on-page markup that
enables search engines to better understand the information
currently on your site’s web pages, and then use this information
to improve search results listing by better matching user intent.

This structured data is defined by using schema to act as the
interpreter. This is the definition we add to the page using
schema code.
Google allows 3 types.
• RDFa
• Microdata
• JSON-LD PREFERS

Schema
JSON-LD is the recommended schema code.
JSON-LD stands for JavaScript Object Notation for Linked Data
This is just a way to implement schema outside the HTML mark-up
structure. RDFa and Microformats required the code to be implemented
via HTML.

Schema
Benefit is it can be removed from the HTML structure, which
makes it easier to write, implement, and maintain.
Resources.
For a good breakdown on what JSON is at the code level.
Portent’s JSON Implementation Guide is very helpful.
https://www.portent.com/blog/seo/json-ld-implementation-guide.htm
And Google has a section in the Developer Guides
https://developers.google.com/search/docs/guides/intro-structured-data

JSON-LD Schema

Schema
IMPORTANT! Test your JSON-LD.
Use the Google Structured Mark-Up Helper.
https://search.google.com/structured-data/testing-tool

Schema
NOTE this tool only tells you if it is semantically correct, NOT if
you are using the proper schema.
Make sure to check with Google’s Guides on schema implementation.
Improper use or implementation can result in a manual action.
• https://developers.google.com/search/docs/guides/intro-structured-data
• https://developers.google.com/search/docs/guides/prototype

Schema
IMPORTANT! Your JSON
content MUST match what is in
the page exactly.
If they differ, you will likely get a
manual action as Google sees
this as cloaking.

Why Does Schema Matter?

We can act as the interpreter and help “teach”
Google what our site is about.

Adding semantic mark-up
(structured data via schema) allows us to tell
Google what WE SAY our site is about and WHAT
RELATIONSHIPS we define within it.

We can act as the interpreter and help “teach”
Google the context of our content.

We can help give Google a clearer understanding.
That helps us help Google better answer
the questions users ask
and to better surface our content for those users
We give our data meaning
Google Understands

Ranking Without Links

Rank Brain

Rank Brain is used for Unknown Queries where entity
meanings/relationships are unclear or unknown.

Rank Brain.
One of two algorithms that uses AI
on the live results
Rank Brain.

Presence of Rank Brain means
Google is confused …

Why?
Google does not use true NLP in Search
(Natural Language Processing)
Rank Brain.

• Words go in.
• Words get assigned a mathematical address in a vector.
• Similar and related words sit close to each other in the vector space.
• Words are retrieved based on your query and the words it locates in the “best fit” vector.
• These word “interpretations” are used to return results.
• If the relationships are weak or unknown, enter Rank Brain.
• Behind the scenes, data is continually fed into the machine learning process, so as to make
those results more relevant the next time.
Rank Brain – Known Relationships.

Rank Brain Also Uses Users Queries & Clicks
to Help It Understand Query Intent.

Remember this result?

A year later …
(yellow = poor intent match)

Google + Rank Brain Also Uses Users Queries & Clicks
to Help It Understand Query Intent.
+ GEO LOCATION

A year later in London …

Why location?
Semantic Relevancy

Why location?
Semantic Relevancy
Sweets has no definitive entity in the US.

Neural Matching
Rank Brain has a friend now.

Rank Brain.
What is the difference between
Rank Brain and Neural Matching?
Rank Brain.

Rank Brain vs Neural Matching.
Rank Brain = concepts
Neural Matching = linking words to concepts
“…neural matching, – AI method to better connect words to concepts.” - Google

Rank Brain vs Neural Matching.
A Google patent related to Rank Brain and Neural Matching
describes a system that uses traditional ranking factors to decide
what is relevant, but NOT what is in the top 10.
Which may be re-ordered post retrieval according to
“ad hoc retrieval” methods and ”dynamic relevancy”
https://www.searchenginejournal.com/google-neural-matching/271125/

AND IT DOES
NOT USE LINKS
https://www.oncrawl.com/technical-seo/neural-matching-seo-content-creation-rules/

So should you optimize for Rank Brain and
Neural Matching?
Rank Brain and Neural Matching.

Why would you optimize to rank with AI?
AI Is ever-changing and unfixed.
Don’t waste the time and resources.
Rank Brain.

Google Does Not Even Understand What
Rank Brain (and neural matching) is Actually Doing.
Rank Brain and Neural Matching.

(Gary Illyes)
Rank Brain.

Simple answer to a very complex issue?
Do your normal query research then just write in
natural and conversational language.
Create holistic content.

Holistic Content?

Write holistic content.
Use terms that are semantically related.
For a detailed explanation Google explains here > https://www.youtube.com/watch?v=vzoe2G5g-w4&feature=youtu.be&t=32m19s

Write holistic content.
DOES YOUR CONTENT HAVE DEPTH AND WIDTH?
For a detailed explanation Google explains here > https://www.youtube.com/watch?v=vzoe2G5g-w4&feature=youtu.be&t=32m19s

And Use Well Formed Text.

Well Formed Text.
http://www.kurzweilai.net/google-open-sources-natural-language-understanding-tools
Ray Kurzweil on Google NLU

Questions = Well Formed Text
https://ai.google/research/pubs/pub47323

THINK in Intent, Query Terms, & Context.

And Questions!

Takeaways.
• Think Search Queries NOT Simple Keywords
• Write in natural, conversational language
• Write using holistic content
• Focus on depth and breadth with related terms
• Add Structured Data
• Use well formed text (ie questions) when you can.
Takeaways.

Google is an Answer Engine.

https://cloud.google.com/natural-language/

Google Natural Cloud Tool

BONUS.

https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0

The Death of the Keyword: In Search of NLP -- Presented at Ungagged London Apr 2019.

More Related Content

What's hot (17)

Similar to The Death of the Keyword: In Search of NLP -- Presented at Ungagged London Apr 2019. (20)

More from Kristine Schachinger SEO and Online Marketing (19)

Recently uploaded (20)

The Death of the Keyword: In Search of NLP -- Presented at Ungagged London Apr 2019.