Linkanalysis handout

Link Analysis

Paolo Boldi
DSI
LAW (Laboratory for Web Algorithmics)
Universit` degli Studi di Milan
a

Paolo Boldi Link Analysis

Ranking, search engines, social networks

Ranking is of uttermost importance in IR, search engines and also
in other social networks (e.g., facebook):

Choosing which of your friends’ signals are relevant for you?
Choosing which of your non-friends should be suggested as
new contact?

In traditional information retrieval, ranking is typically realized
through a scoring system:

σ :D×Q→R

that assigns a “relevance” score to every document/query pair.
Rankings may be composed (e.g., by linear combination): this is
called rank aggregation.


Web Search

What happens when a search engine receives a certain query q
from a user?
Selection: it selects, from the set D of all available
documents, a subset S(q) of documents that satisfy q;
Ranking: it establishes a total order on S(q) determining how
the results should be presented to the user.


Ranking

A most crucial step!
Typical queries have just one or two words (recent survey says
3.08 on average), and million of results
The typical user just looks at the ﬁrst result page; often, just
the ﬁrst link (Feeling lucky)
In the past, a search engine’s share of market used to depend
on freshness, usability, coverage, additional features. . .
. . . now, the competitive edge is determined mostly by ranking!


Link Analysis — Problem and assumptions

Static Ranking problem: Assign to each web page a score
that is proportional to its importance. Use only linkage
structure to this aim.
Basic assumption: A link is a way to confer importance.


The Web as a Graph

You can think of the Web as a (directed) graph:
its nodes are the URLs
there is an arc from node x to node y iﬀ the page with URL x
contains a hyperlink towards URL y .
This is called the Web graph.


How Large Is It?

It is extremely diﬃcult to evaluate the Web size!
about 8/11 billions of nodes (excluding the dark mass)
about 300/500 billions of arcs
about 400/600 millions of hosts
about 2,000 millions of users.
Dark mass = pages that cannot be reached: according to prudent
estimates, currently indexed web is just the 16-20% of the “real”
web!
BE CAREFUL! Our knowledge of the Web is only indirect! (it is
what we can collect using web crawlers, or interrogating search
engines. . . )


What Is the Shape of the Web?

The (known part of the) Web graph has a bowtie-like
shape. . .
Subdomains (e.g., country-code domains [.fr, .it, . . . ],
large intranets etc.) are similar: a fractal-like graph
It is highly dynamic



Giant component (core)
A strongly-connected component containing about 30% of the
pages
Estimated diameter: directed → 20/30; undirected → 10/17
“It’s a small-world!”



Left-hand side
Ancestors of the giant component in the scc DAG
About 25%
Estimating its size is very diﬃcult (how can you get there???)
They are the Web pariahs: they are there, but no one wants
to link them!



Right-hand side
Descendants of the giant component in the scc DAG
About 25%
Contains all “documents” with no hyperlinks in them (e.g.,
pure text, word/PDF/PostScript with no links etc.)



Tubes, tendrils and isolated components
Remaining nodes (20%)


Ranking Techniques: A Taxonomy

Depending on whether the scoring (ranking) function depends or
not on the query, and whether it depends or not on the text of the
page (or only on its links):

Query-dependent (dynamic) Query-independent (static)
Text-based IR (already treated) -
Link-based HITS PageRank


PageRank [Brin, Page, 1998]

An extremely popular ranking technique, because. . .
it is static, so it can be computed beforehand (not at query
time)
it can be computed eﬃciently
it is (used to be) the main ranking technique used at Google.


PageRank — An introductory metaphor (1)

Every page has an amount of money that, at the end of the
game, will be proportional to its importance.
At the beginning, everybody has the same amount of money.
At every step, every page x gives away all of its money,
redistributing it equally among its out-neighbors.

Problem with this solution: Formation of oligopolies that “suck
away” all money from the system, without ever giving it back.


PageRank — An introductory metaphor (2)

At every step, only a ﬁxed fraction α < 1 of the money a page
has is redistributed to its neighbors; the remaining fraction
1 − α is paid to the state (a form of taxation).
The state redistributes the money collected to all nodes,
according to a certain preference vector v (e.g., the uniform
distribution, the “Berlusconi” distribution. . . ).

Another problem: What should the dangling nodes do? (A
dangling node is one that has no out-neighbors)
Dangling nodes pay, as every other node, 1 − α in taxes, and
distribute α to the nodes according to a ﬁxed dangling-node
distribution u.


PageRank: the Web-Surfer Metaphor

0 1

2 4 5

6 7 8 9

3
A surfer is wandering about the web. . .

0 1

What does PageRank depends on?

PageRank can be formally deﬁned as the limit distribution of a
stochastic process whose states are Web pages.
What does this distribution depend on? (more on all this later)
the web graph G ;
the preference vector v;
the dangling-node distribution u;
the damping factor α.
How does PageRank depends on each of these factors? What
happens at limit values (e.g., α → 1)?


PageRank: formal definition

Is the definition of PageRank well-given? Are we all using the
same definition?
The row-normalised matrix of a (web) graph G is the matrix
¯ ¯
G such that (G )ij is one over the outdegree of i if there is an
arc from i to j in G (in general, and usually, not stochastic
because of rows of zeroes).
d is the characteristic vector of dangling nodes (nodes without
outgoing arcs).
Let v and u be distributions, which we will call the preference
and the dangling-node distribution.
Let α be the damping factor.


PageRank: formal deﬁnition (2)
PageRank r is deﬁned (up to a scalar) by the eigenvector
equation
¯
r α(G + dT u) + (1 − α)1T v = r

Equivalently, as the unique stationary state of the Markov
chain
¯
α(G + dT u) + (1 − α)1T v
that we call a Markov chain with restart [Boldi, Lonati,
Santini & Vigna 2006].
Some notation:
¯
r α (G + dT u) + (1 − α)1T v = r

Some notation:
r α P + (1 − α)1T v = r


PageRank closed formula
Fixing r1T = 1,
rM = r
T
r αP + (1 − α)1 v = r
αrP + (1 − α)v = r
(1 − α)v = r(I − αP),
. . . which yields the following closed formula for PageRank:
r = (1 − α)v(1 − αP)−1 .
So it’s a linear system—use Gauss–Seidel!
Or use the Power Iteration Method:
lim x · M k
k→∞

Equivalently:
∞
r = (1 − α)v (αP)k .
k=0

PageRank and graph paths
Let G ∗ (−, i) be the set of all paths ending into i;
For any π ∈ G ∗ (−, i), let b(π) denote the branching
contribution of π, i.e., the product of outdegrees of the nodes
that are met on the path (excluding the ending node);
The expression
∞
r = (1 − α)v (αP)k ,
k=0
can be rewritten as
vs(π) |π|
(r)i = (1 − α) α
b(π)
π∈G ∗ (−,i)

The expression
∞
r = (1 − α)v (αP)k ,
k=0
can be rewritten as Paolo Boldi Link Analysis

Iteration vs. approximation
We can rewrite the summation as follows:
∞
r =v+v αk P k − P k−1 .
k=1

Thus, the rational function r can be approximated using its
Maclaurin polynomials (i.e., truncated series).
Theorem
The n-th approximation of PageRank computed by the Power
Method with damping factor α and starting vector v coincides with
the n-th degree Maclaurin polynomial of PageRank evaluated in α.
n
vM n = v + v αk P k − P k−1 .
k=1


One α to rule them all. . .

Corollary
The difference between the k-th and the (k − 1)-th approximation
of PageRank (as computed by the Power Method with starting
vector v), divided by αk , is the k-th coefficient of the power series
of PageRank.
As a consequence the data obtained computing PageRank for a
given α can be used to compute immediately PageRank for any
other α, obtaining the result of the Power Method after the same
number of iterations.
By saving the Maclaurin coefficients during the computation of
PageRank with a specific α it is possible to study the behaviour of
PageRank when α varies.
Even more is true, of course: using standard series derivation
techniques, one can approximate the k-th derivative.


Some typical behaviours

1e-07 2.6e-08

9e-08 2.4e-08

2.2e-08
8e-08
2e-08
7e-08
1.8e-08
6e-08
1.6e-08
5e-08
1.4e-08
4e-08
1.2e-08

3e-08 1e-08

2e-08 8e-09
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

5.5e-08 3.5e-08

5e-08
3e-08
4.5e-08
2.5e-08
4e-08

3.5e-08 2e-08

3e-08
1.5e-08
2.5e-08
1e-08
2e-08

1.5e-08 5e-09
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1


An example
0.5
node 0
node 1
node 2
node 3
0.4 node 4
0 1 node
node
5
6
node 7
node 8
0.3 node 9

2 4 5
0.2
6 7 8 9

3 0.1

0
0 0.2 0.4 0.6 0.8 1

(−1 + α) α2 + 18 α + 4
r0 (α) = −5
8 α4 + α3 − 170 α2 − 20 α + 200
(−1 + α) α2 + 2 α + 10
r1 (α) = −2
8 α4 + α3 − 170 α2 − 20 α + 200

Strong vs. weak
Clearly, the preference vector conditions significantly
PageRank, but. . .
. . . in real-world crawls, which have a large number of
dangling nodes, the dangling preference is also very important.
In the literature one can find several alternatives (e.g., u = v
or u = 1/n).
We suggest to distinguish clearly between strongly preferential
PageRank (u = v) and weakly preferential PageRank.
Papers abound on both sides (and even on the
I-don’t-care-about-dangling-nodes side!). . .
. . . but the two versions are very different!
By “very different” we mean both in the resulting ordering
and in the mathematical properties.
On a 100 million pages snapshot of the .uk domain, Kendall’s
τ between strong and weak is ≈ .25 for a topic-based v and
u = 1/n! [Boldi, Posenato, Santini & Vigna 2006]

The magic value α = 0.85

One usually computes and considers only r(0.85). Why 0.85?
“The smart guys at Google use 0.85” (???).
“It works pretty well”.
Iterative algorithms that approximate PageRank converge
quickly if α = 0.85: larger values would require more
iterations; moreover. . .
. . . numeric instability arises when α is too close to 1. . .
. . . yet, we believe that understanding how r(α) changes when
α is modiﬁed is important.


Some literature

PageRank (values and rankings) change signiﬁcantly when α
is modiﬁed [Pretto 2002; Langville & Meyer 2004].
Convergence rate of the Power Method is α [Haveliwala &
Kamvar 2003].
The condition number of the PageRank problem is
(1 + α)/(1 − α) [Haveliwala & Kamvar 2003].
PageRank can be computed in the α ≈ 1 zone using
Arnoldi-type methods [Del Corso, Gull` & Romani 2005;
ı
Golub & Grief 2006].
PageRank can be extrapolated when α ≈ 1 (even α > 1!)
using an explicit formula based on the Jordan normal form
[Serra–Capizzano 2005; Brezinski & Redivo–Zaglia 2006]
Choose α = 1/2! [Avrachenkov, Litvak & Kim 2006]
. . . and many others.


What happens when α → 1?

lim M = P.
α→1

The “preferential” part added to P vanishes, whereas the part due
¯
to G and u becomes larger: some interpret this fact as a hint that
r becomes “more faithful to reality” when α → 1.
Is this true?
Since r is a coordinatewise bounded function deﬁned on [0, 1), the
limit
r∗ = lim r
α→1−

exists.


A ready-made solution
In fact, since the resolvent (I /α − P) has a Laurent expansion
around 1 in the largest disc not containing 1/λ for another
eigenvalue λ of P, PageRank is analytic in the same disc; a
standard computation yields
∞ n+1
α−1
(1 − α)(1 − αP)−1 = P ∗ − Q n+1 ,
α
n=0

where Q = (I − P + P ∗ )−1 − P ∗ and
n−1
∗ 1
P = lim Pk
n→∞ n
k=0
is the Ces´ro limit of P.
a
We conclude that
r∗ = vP ∗ .
What makes r∗ diﬀerent from other limit distributions? How can
we describe its structure?

Characterising r∗

We shall characterise r∗ using the structure of G (even in the
presence of dangling nodes).
A node x of G is a bucket iﬀ it is contained in a non-trivial
strongly connected component with no arcs toward other
components. (Non-trivial means that it contains at least one arc)


A characterisation theorem

Corollary
Assume u = 1/n. Then:
1. if G contains a bucket then a node is recurrent for P iff it is a
bucket;
2. if G does not contain a bucket all nodes are recurrent for P.

Theorem
1. If a bucket of G is reachable from the support of u then a
node is recurrent for P iff it is a bucket of G ;
2. if no bucket of G is reachable from the support of u, all nodes
reachable from the support of u form a bucket component of
P; hence, a node is recurrent for P iff it is in a bucket
component of G or it is reachable from the support of u.


Bowtie

As a consequence, when α → 1, all PageRank concentrates in a
bunch of pages that live in the rightmost part of the bowtie
[Kumar et al., ’00]:

r(α) becomes meaningless as α → 1!


Interpretation

The statement of the previous theorem may seem a bit
unfathomable. The essence, however, could be stated as follows:
except for strongly connected graphs, or graphs whose terminal
components are dangling, the recurrent nodes are exactly the
buckets (unless we are in the very pathological case in which no
bucket is reachable from the support of u).
As we remarked, a real-world graph will certainly contain many
buckets, so the ﬁrst statement of the theorem will hold. This
means that most nodes x will have zero rank when α → 1;
particular, all nodes in the core component.
In a word: PageRank when α → 1 is nonsense in all real-world
cases. . .
. . . and if you want the dire truth, there is an explicit formula in
[Avrachenkov, Litvak & Kim 2006].


General behaviour
What about the general behaviour of r?
We have an explicit formula for derivatives of PageRank (k > 0):

r(k) (α) = k!v P k − P k−1 (I − αP)−(k+1) .

Approximating them is also not diﬃcult, since we have Maclaurin
polynomials ( r(k) (α) t is the polynomial of order t):
Theorem
If t ≥ k/(1 − α),

δt
r(k) (α) − r(k) (α) t ≤ r(k) (α) t − r(k) (α) t−1 ,
1 − δt
where
α(t + 1)
1 > δt = .
t +1−k


An alternative proposal. . .

Instead of using a speciﬁc value of α, one could try to use the
average value, or equivalently:
1
Ti = (r)i dα (TotalRank [Boldi 2005])
0

Also TotalRank is a special case of the general ranking technique
of [Baeza–Yates, Boldi & Castillo 2006]. The two damping
functions for TotalRank and PageRank are:
1
dT ( ) =
(t + 1)(t + 2)
dP ( ) = (1 − α)α .


. . . and a possible explanation for .85

If you consider the sum of their diﬀerences up to length (average
path length in the graph you are considering), you get:

+1 1
α − .
+2

For a given , the value α∗ ( ) minimizing this sum is:

log log2
α∗ ( ) = 1 − +O 2
.

The average path length of the Web is about 20, and
α∗ (20) ≈ .85. . .


So you know about α. . .

. . . but what about the dependence from u and v?
Clearly, weakly preferential PageRank is a linear operator
associating to the preference distribution another distribution. Said
otherwise, for a ﬁxed α PageRank is a linear function.
This linear dependence makes it possible to compute directly
PageRank on any convex combination of preference vectors
for which it is already known.
This property is essential to compute personalised scores [Jeh &
Widom 2002].
Using the Sherman–Morrison formula it is possible to make the
dependence on v and u explicit, and sort out what happens in the
strongly preferential case.


Pseudoranks

Let us define the pseudorank of G with preference vector v and
damping factor α ∈ [0 . . 1]:

¯ −1
v(α) = (1 − α)v I − αG .

The above definition can be extended by continuity to α = 1 even
¯
when 1 is an eigenvalue of G , always using the fact that
¯ has a Laurent expansion around 1, getting again vG ∗ .
I /α − G ¯
¯
When α < 1 the matrix (I − αG ) is strictly diagonally dominant,
so the Gauss–Seidel method can be used to compute quickly
pseudoranks.
Note that v(α) is linear in v.
The notion appears in [Del Corso, Gull` & Romani 2004] and it has
ı
been used in [McSherry 2005; Fogaras, Rćz, Csalogńy & Sarl´s
a a o
2005] (actually, as the definition of PageRank).


Explicit dependence

Using pseudoranks we can easily express the dependence [Boldi,
Posenato, Santini & Vigna 2006]:

v(α)dT
r = v(α) − u(α).
1 − α + u(α)dT
1

Using this formula, once the pseudoranks for certain distributions
have been computed, it is possible to compute PageRank using
any convex combination of such distributions as preference and
dangling-node distribution.
Another evident feature of the above formula is that the
dependence on the dangling-node distribution is not linear, so we
cannot expect strongly preferential PageRank to be linear in v.


The strongly preferential case

Nonetheless, if we ﬁx u = v and simplify the resulting formula
(getting back the formula obtained by Del Corso, Gull` and
ı
Romani). . .
v(α)dT
r = v(α) 1 −
1 − α + v(α)dT
1

So pseudoranks are just multiples of strongly preferential ranks,
and the side eﬀect is that strongly preferential PageRank can be
computed by convex combination of pseudoranks.
Assuming that v = λx + (1 − λ)y, we have

r = rλx+(1−λ)y (α) ∝ λx(α) + (1 − λ)y(α)


Alternatives to PageRank

PageRank is but one of the many link-based methods to establish
page importance. Other notable examples are:
HITS (Kleinberg)
SALSA (Lempel, Moran), a variant of HITS (not covered
here)


HITS

HITS (Hyperlink-Induced Topic Search) is based on the idea that
the web contains, for every topic, two “types” of pages:
authoritative pages about the topic
hub pages that are not authoritative but contain link to many
authoritative pages.
HITS gives two scores to every page, measuring their
authoritativeness and their hubbiness.
Diﬀerently from PageRank it is not query independent.


HITS (cont’d)

The algorithm works in two phases:
a graph Gq (a subgraph of the whole web graph) is singled
out (depending on the query)
the authoritativeness/hubbiness scores are computed for the
pages in Gq


HITS — Phase 1

Gq is obtained as follows:
the set Sq of the top k pages relative to q are obtained using
some techniques (e.g., BM25)
for each x ∈ Sq , all nodes in N + (x) are added
for each x ∈ Sq , at most h nodes of N − (x) are added


HITS — Phase 2

At every iteration, we will have two scores hx (t) and ax (t) for
every node x ∈ NGq .

hx (t + 1) ∝ ay (t)
x→y

ax (t + 1) ∝ hy (t)
x←y

The ∝ is necessary to avoid divergence (the scores are normalized
at every iteration).


HITS — In practice

HITS (proposed by Kleinberg in 1999) is not used by most search
engine, probably due to:
its dynamic nature (requiring computation at query time)
its marginal beneﬁts over PageRank.
It was supposedly used by Teoma (later Ask.com).


Linkanalysis handout

More Related Content

Similar to Linkanalysis handout (20)

More from csedays (8)

Linkanalysis handout