![2 C. Mallows
algorithms are copied directly from implementations in the S language. Most of the
S functions that we use are self-explanatory, but a detailed explanation appears in
Appendix 1. Only two things need explanation here; the function c() (concatenate)
makes its arguments into a vector. Also, many S functions take a vector argument.
It is convenient that subscripts are not used in S; indices are shown by using square
brackets. Thus a vector x of length 3 has elements x[1],x[2],x[3]. This notation
makes it easy to write complicated expressions as indices.
3. Two naive methods, and a new idea
Recall that the observed samples are x = c(x[1], . . . , x[m]) and z = c(z[1], . . . , z[n]).
If we have m = n, a first suggestion is to sort x and z, forming sortx and sortz,
and to form yhat ← sortz − sortx (i.e. yhat[i] = sortz[i] - sortx[i]). If the
distributions of X and Z are Normal with variances σ2
and τ2
respectively, so that
what we want is an estimate of a Normal distribution with variance τ2
− σ2
, this
method produces an estimate of a Normal distribution with the correct mean but
with variance (τ − σ)2
(because the sorted vectors are perfectly correlated), which
is too small. The method is not consistent as n → ∞.
Another approach, still assuming m=n, is to put both x and z into random orders
and to compute the vector of differences z-x. Again, this does not work; this gives
an estimate of the distribution of X + Y − X
where X
is an independent copy of
X. In the Normal case described above, this method gives an estimate of a Normal
distribution with variance τ2
+ σ2
instead of τ2
− σ2
.
The new idea is that a useful estimate could be obtained if we knew the “right”
order in which to take the zs before subtracting the xs; and we can estimate an
appropriate order by a simulation. Here is a first version of how this would work,
assuming m=n. Suppose we have a first estimate of FY , represented by a vector of
values oldy = c(oldy[1],...,oldy[n]). We choose a random permutation rperm
of (1, . . . , n), and put the elements of oldy into this order. We add the xs to give a
vector w where
w ← x + oldy[rperm]
We record the ranks of the elements of this vector. We put the elements of z into
this same order and subtract the xs. Thus
newy ← sort(z)[rank(w)] − x
We can repeat this operation as many times as we like.
We will attempt an explanation of why this might be expected to work below.
An example is shown in Figure 1. Here the sample size is n = 100, and both X and
Y are standard Normal. We generated pseudo-random samples z0 = x0 + y0 and
x1, placed these in sorted order (sortz0 = sort(z0) and sortx1 = sort(x1) and
started the algorithm by taking y1 = sort(sortz0 - sortx1). Successive versions
of y were obtained using the iteration. Note that rank(runif(n)) is a random
permutation of (1,. . . ,n).
newy ← sort(sortz0[rank(sortx1 + oldy[rank(runif(n))])] − sortx1).
We ran the iteration for 100 steps. Figure 1 shows the first nine y vectors, each
sorted into increasing order, plotted against standard normal quantiles. Also shown
is the straight line that corresponds to a normal distribution with mean mean(z0)
- mean(x1) and variance var(z0) - var(x1). Figure 2 shows iterations 81:100.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-14-320.jpg)
![Deconvolution 3
Fig 1. QQplots of the first nine iterations for the Normal example.
Figure 3 shows values of a distance index d, which is the sum of absolute vertical
deviations between this line and the estimate y. The algorithm appears to be stable,
meaning that in repeated applications of the algorithm, the estimates stay close
together. The initial transient takes no more than four iterations. The average value
of the distance d over iterations 5:100 is 19.69. Also shown (with plotting character
“o”) are comparable values of d for random samples from a normal distribution
with the same mean and variance as this fitted normal distribution. The average
value of these is 14.74. If we average the y vectors over iterations 5:100, we get
a vector whose distance from this fitted normal distribution is d = 17.82. The
average distance between the iterates and their average is d = 8.95. Thus the
average distance between the iterates and their average is smaller than the average
distance between random normal samples and the population line.
The iteration seems to be giving good estimates of Y . Why should this be so?
Here is an argument to support this expectation. Suppose z = x+y; these vectors
are realizations of random variables X, Y, Z. we cannot observe any of x,y,z but
can see sortz = sort(z) and an independent realization of X, namely x1. How
can we define an estimate of y? Since X and Y are independent (by assumption),
if rperm is a random permutation, then zhat = sort(x) + sort(y)[rperm] is a
realization of Z, sorted according to sort(x). To retrieve y we simply subtract
sort(x) from zhat. If n is large, we expect zhat to be close to z, and sort(x1) to
be close to sort(x0). Thus we expect that putting z into the same order as zhat
will make z approximately equal to zhat; and subtracting sort(x1) from this will
approximately retrieve y. This argument does not explain why the iteration should
converge when it is started with y0 remote from the correct value. We do not yet
have an explanation of this.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-15-320.jpg)
![4 C. Mallows
Fig 2. QQplots of iterations 81:100 for the Normal example.
4. Questions
Several questions come to mind immediately. Is this algorithm always stable? Is the
algorithm consistent, meaning that as n ← ∞, the empirical c.d.f of y converges in
probability to FY ? I thank a referee for reminding me that FY may not be unique.
What happens when it is not?
To approach these questions, we point out that in the algorithm we have de-
scribed, the possible values of the vector y are all of the form z[perm] − x where
perm is a permutation of (1, . . . , n). Thus in repeated applications y executes a
random walk on the n! possible values of this vector. This random walk will have a
stationary distribution, which may not concentrate on a single state (this seems to
be the usual case). Some states may be transient. Thus the most we can hope for
is that this stationary distribution is close to FY in some sense.
Clearly we need a proof that as n → ∞ this stationary distribution converges (in
some sense) to a distribution that is FY whenever this is identifiable. Also it would
be very pleasant to understand the distribution of the discrepancy measure d when
y is drawn from the stationary distribution. As yet we do not have these results, but
empirical evidence strongly suggests that the convergence result holds universally,
and that useful estimates are obtained in all cases. However the dispersion among
successive realizations of y is an over-optimistic estimate of the precision of the
estimate of FY .
Detailed analysis of the stationary distribution seems out of reach. Even with m
= n = 3, 924 different configurations of x and z need to be considered. There are
208 distinct stationary distributions. See Appendix 2.
We suggest that in practice we need to ignore an initial transient, and that the](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-16-320.jpg)
![Deconvolution 5
Fig 3. The index d for the first 100 iterations, with values for random normal samples.
dispersion among successive realizations of y is an over-optimistic estimate of the
precision of the estimate of FY .
We need to consider how to handle boundary conditions, for example (as in
the motivating example) that all values of Y are positive. The algorithm as stated
need not generate vectors y that satisfy such conditions. Also, we question how
the algorithm will perform when there are remote outliers in either or both z0 and
x1. Since these samples are assumed to be independent of one another, there is no
reason to hope that subtracting an x1 outlier from a z0 outlier will make any sense.
We study these questions in Section 6 below.
5. Variations
Several variations on the basic idea are as follows.
(a) Instead of using the actual data (x[1],...,x[n]) use a sample from an
estimate of FX, for example a bootstrap sample from the observed x.
(b) To add some smoothness to the algorithm, at each iteration replace x by x + ξ
and/or y by y + η, where ξ and η are vectors of small Gaussian perturbations.
We can use the same perturbations throughout, or we can use independent
perturbations at each step of the algorithm.
(c) Similarly we can (independently) smooth z by adding ζ. If we arrange that
varζ = varξ + varη, these smoothings should not introduce any bias into the
estimate of FY , because X + ξ + Y + η is distributed like Z + ζ. Of course
the efficiency of the method will degrade if the variance of ζ becomes large
(unless each of X, Y, Z is Gaussian).](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-17-320.jpg)
![Deconvolution 7
Fig 4. The lowest 20 elements of the first nine iterations for each of three methods: Top:(q),
Middle:(l), Bottom:(p).
(p) Remove negative values of yhat, replacing them by their absolute values. This
can be done in S by an application of the abs function:
newy ← sort(abs(sortz0[rank(sortx1
+oldy[rank(runif(n))])] − sortx1)).
All three methods performed similarly for values of yhat greater than 0.25.
Figure 4 shows the lowest 20 values of yhat for the first nine iterations, plotted
against standard exponential quantiles, for each of these three methods, together
with the line through the origin with slope mean(z0) - mean(x1). We see that the
naive method (q) produces a large number of negative values; method (l) avoids
this but seems to introduce a positive bias; method (p) works well.
Figure 5 shows the number of negative elements in yhat (before adjustment) for
the three methods. The average numbers over the first 100 iterations are (q) 4.44,
(l) 3.35, (p) 2.68. We have no understanding why the “absolute values” method
works as well as it seems to.
We have run similar trials for the case where both X and Y are uniform on
(0, 1), so that Z has a triangular density supported on (0, 2). Here for method (p)
we need to reflect values above y=1 to lie in (0, 1). Again, method (p) seems to be
better than (q) and (l).
We have also investigated the performance of the “absolute values” method when
there is a positivity condition and outliers are present. We find that the non-outlying
part of the distribution is estimated satisfactorily. We took x0, x1, and y0 each to
contain 95 samples from a standard exponential distribution (with mean 1), and](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-19-320.jpg)
![8 C. Mallows
Fig 5. The number of negative elements (before adjustment) for each of three methods: Top:(q),
Middle:(l), Bottom:(p).
5 samples from an exponential distribution with mean 100. Figure 6 shows the
first four iterations of our basic algorithm, using the option (p) to adjust negative
estimates, plotted against sort(y0). Figure 7 expands the lower corner of this plot,
with the line through the origin of unit slope. The iterates seem to be staying close
to this line.
At this point our recommendation (if m = n and the variables are continuous,
so that there are no ties in the computed values), is to use the original method,
i.e. do not bootstrap or smooth or use (j). If the variables are lattice-valued, for
example integer-valued, it seems to help to add small random perturbations to x
and y[rperm] at each stage to break the ties randomly. It is not clear whether it is
as good to simply add small perturbations once and for all. To handle the boundary
and outlier problems, we recommend using the absolute-values method (p) above.
Appendix 1. The S language
In S the basic units of discourse are vectors; most functions take vector arguments.
The elements of a vector x of length n are x[1], . . . , x[n]. c() is the “concatenate”
function, which creates a vector from its arguments. Thus x = c(x[1], . . . , x[n]). If
the elements of an m-vector s are drawn from 1, . . . , n, (possibly with repetitions),
x[s] is the vector c(x[s[1]], . . . , x[s[m]]. The function sort() rearranges the ele-
ments of its argument into increasing order; so if x = c(2,6,3,4), sort(x) is
c(2,3,4,6). The function rank() returns the ranks of the elements of its argu-
ment; i.e. rank(x)[i] is the rank of x[i] in x. Thus if x = c(2,6,3,4), rank(x)](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-20-320.jpg)
![Deconvolution 9
Fig 6. Four iterates of the basic algorithm, using option (p) to handle the positivity condition,
when outliers are present.
is c(1,4,2,3). sort(x)[rank(x)] is just x. Another function we have used is
runif(), which generates pseudo-random uniform variables drawn from the in-
terval (0,1). Thus rank(runif(n)) is a random permutation of 1, . . . , n rnorm(n)
generates n standard normals; rexp(n) generates n random exponentials. The func-
tion abs() replaces the elements of its argument by their absolute values.
Appendix 2. The case m=n=3
Without loss of generality we may assume x1 = c(0,x,1) and z0 = c(-a,0,b)
with 0 x 1/2 and a and b positive. Examination of the 36 possible values of
(z0[perm1]-x1)[perm2] +x1 shows that the stationary distribution will change
whenever any of a,b and a+b crosses any of the values x,2x,1,1+x,1-x,1-2x,2,
2-x,2-2x. For a general x in (0,1/2) these cut-lines divide the positive quadrant
of the a,b plane into 154 regions. The configuration of these regions changes when
x passes through the values (1/6,1/5,1/4,1/3,2/5). Thus we need to consider six
representative values of x, perhaps x = c(10,22,27,35,44,54)/120, and for each
of these values of x we have 154 regions, 924 regions in all. We computed the
transition matrix of the random walk for each of these 924 cases, and found 208
different stationary distributions. One of these, where one state is absorbing and
the other five transient, occurs 84 times. Ten distributions occur only once each.
A similar calculation for m or n larger than 3 seems impractical.
Acknowledgments. Thanks to Lorraine Denby, for showing me the problem,
and to Lingsong Zhang, who did some of the early simulations. Also to Jim Landwehr,](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-21-320.jpg)
![10 C. Mallows
Fig 7. Expansion of the lower corner of Figure 6.
Jon Bentley and Aiyou Chen for stimulating comments. Two referees contributed
insightful remarks.
References
[1] Carroll, R. J. and Hall, P. (1988). Optimal rates of convergence for
deconvolving a density. J. Amer. Statist. Assoc. 83 1184–1186.
[2] Fan, J. (1991). On the optimal rates of convergence for nonparametric decon-
volution problems. Ann. Statist. 19 1257–1272.
[3] Fan, J. (2002). Wavelet deconvolution. IEEE Trans. Inform. Theory 48 734–
747.
[4] Goldenshluger, A. (1999). On pointwise adaptive nonparametric deconvo-
lution. Bernoulli 5 907–925.
[5] Jansson, P. A., ed. (1997). Deconvolution of Images and Spectra. Academic
Press, New York.
[6] Liu, M. C. and Taylor, R. L. (1989). A consistent nonparametric density
estimator for the deconvolution problem. Canad. J. Statist. 17 427–438.
[7] Mendelsohn, J. and Rice, J. (1982). Deconvolution of microfluorometric
histograms with B-splines. J. Amer. Statist. Assoc. 77 748–753.
[8] Starck, J.-L. and Bijaoui, A. (1994). Filtering and deconvolution by the
wavelet transform. IEEE Trans. Signal Processing 35 195–211.
[9] Stefanski, L. A. and Carroll, R. J. (1990). Deconvoluting kernel density
estimators. Statistics 21 169–184.
[10] Zhang, C. H. (1990). Fourier methods for estimating mixing densities and
distributions. Ann. Statist. 18 806–830.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-22-320.jpg)
![Deconvolution 11
[11] Zhang, H. S., Liu, X. J. and Chai, T. Y. (1997). A new method for optical
deconvolution. IEEE Trans. Signal Processing 45 2596–2599.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-23-320.jpg)
![IMS Lecture Notes–Monograph Series
Complex Datasets and Inverse Problems: Tomography, Networks and Beyond
Vol. 54 (2007) 12–23
c
Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000030
An iterative tomogravity algorithm for
the estimation of network traffic
Jiangang Fang1
, Yehuda Vardi1,∗
and Cun-Hui Zhang1,†
Rutgers University
Abstract: This paper introduces an iterative tomogravity algorithm for the
estimation of a network traffic matrix based on one snapshot observation of
the link loads in the network. The proposed method does not require complete
observation of the total load on individual edge links or proper tuning of a
penalty parameter as existing methods do. Numerical results are presented
to demonstrate that the iterative tomogravity method controls the estimation
error well when the link data is fully observed and produces robust results
with moderate amount of missing link data.
1. Introduction
This paper concerns the estimation of network traffic based on link data. The traf-
fic matrix of a network, which gives the amount of source-to-destination (SD) flow,
is an essential element in a wide range of network administration and engineering
applications. However, in today’s fast growing communication networks, it is of-
ten impractical to directly measure network traffic matrices due to cost, network
protocol and/or administrative constraints, while measurements of the total traffic
passing through certain individual links are more readily available. Thus, the prob-
lem of estimating SD traffic based on link data, called network tomography [10], is
of great interest to communications service providers.
In the network tomographic model [10]
y = Ax,
(1.1)
where y is a vector of traffic loads on links, A = (aij) is a known routing matrix
with elements aij = 1 if link i is in the path for the j-th pair of SD nodes and
aij = 0 otherwise, and x is the SD traffic flow as a vectorization of the traffic
matrix. Here the routing protocol A is fixed. In typical network applications, the
number of links (edges) is of the same order as the number of nodes (vertices) in
the network graph, while the number of SD pairs is of the order of the square of
the number of nodes. Thus, dim(y) dim(x) and the network tomographic model
(1.1) is ill-posed. Vardi [10] identified the ill-posedness of (1.1) as the main difficulty
of network tomography and proposed to estimate the expected traffic flow based on
∗Research partially supported by National Science Foundation Grant DMS-0405202.
†Research partially supported by National Science Foundation Grants DMS-0405202, DMS-
0504387 and DMS-0604571.
1Department of Statistics, Hill Center, Busch Campus, Rutgers University, Piscat-
away, New Jersey 08854, USA, e-mail: alexfang@stat.rutgers.edu; vardi@stat.rutgers.edu;
czhang@stat.rutgers.edu
AMS 2000 subject classifications: 62P30, 62H12, 62G05, 62F10.
Keywords and phrases: network traffic flow, network tomography, Kullback-Leiber distance,
network gravity model, regularized estimation.
12](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-24-320.jpg)
![Iterative tomogravity algorithm 13
independent copies of y by modeling the variance of y. The problem has since being
considered by many research groups. See Vanderbei and Iannone [9] for MLE/EM,
Cao et al. [1, 2] for MLE/EM in the model xj ∼ N(λj, φλc
j) and non-stationarity
issues, Medina et al. [8], Liang and Yu [6] for a more scalable pseudo-likelihood,
Liang et al. [7] for additional direct observations of flow data for selected SD pairs,
and Coates et al. [4] and Castro et al. [3] for surveys with additional references. In
general, these methods require observations of multiple copies of y.
An interesting and noticeable development in the area is the introduction of
(tomo)gravity algorithms and related methods based on a single snapshot of the
network, i.e. one copy of y. Zhang et al. [11] observed that in certain communica-
tions networks (e.g. a backbone network where each node represents a PoP, or point
of presence), almost all the traffic flow is generated by and destined to a known
set of edge nodes which do not serve as intermediate nodes in any SD paths. Thus,
each SD path begins with a source edge node, traverses through an inbound edge
link, an inner network, and then an outbound edge link to a destination edge node.
Under this assumption, the total inbound flow N
(in)
s from a source node s is the
sum of the loads over all the inbound edge links from s and the total outbound flow
N
(out)
d to a destination node d is the sum of the loads over all the outbound edge
links to d. The edge nodes communicate to each other through an inner network
with a directed graph composed of inner nodes and links and a routing protocol,
but the inner nodes does not generate or receive traffic. Moreover, Zhang et al. [11]
observed that for each fixed source node s, the distribution of the inbound traffic
N
(in)
s from s to different destinations d is approximately proportional to the total
outbound loads N
(out)
d these destinations receive. Formally, this is called the gravity
model and can be written in Vardi’s [10] vectorization as
xj =
N
(in)
sj N
(out)
dj
N
, N =
s
N(in)
s =
d
N
(out)
d ,
(1.2)
where sj and dj are respectively the source and destination nodes for the j-th SD
pair,
xj is the corresponding component of the simple gravity solution
x as an
approximation of the vector x in (1.1), and N is the total flow. The gravity model
is best described as
xsd = N(in)
s N
(out)
d /N
(1.3)
with a slight abuse of notation, where
xsd is the traffic flow from source s to des-
tination d in the gravity model, i.e.
xj =
xsj dj . Here, the relationship between the
link data y and the SD traffic flow x is still governed by the tomographic model
(1.1). Due to the additional information provided in the gravity model (1.2) about
the nature of the SD traffic x, the number of unknowns in x is square rooted. Thus,
the ill-posedness of (1.1) is greatly alleviated. In particular, if all link loads are ob-
served, the total inbound flow N
(in)
s and outbound flow N
(out)
d for individual edge
nodes and thus the total traffic N are all available network statistics in the gravity
model. In addition to the gravity solution
x in (1.2), Zhang et al. [11] developed
the simple tomogravity solution
arg min
x
x −
x : Ax = y
(1.4)
and more general tomogravity solutions when the edge nodes are further classified
as “access” or “peering”, while Zhang et al. [12] developed entropy regularized](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-25-320.jpg)
![14 J Fang, Y. Vardi and C.-H. Zhang
tomogravity solution as
arg min
x
y − Ax2
+ φN2
K
x/N,
x/N
,
(1.5)
where K(·, ·) is the Kullback-Leibler information and φ is a tuning parameter for
the penalty level. These tomogravity solutions require complete knowledge of the
total inbound and outbound flow, i.e. N
(in)
s and N
(out)
d , for all individual source and
destination nodes. They perform reasonably well when such information is available
and have been implemented in certain ATT commercial networks.
In this paper, we propose an iterative tomogravity (ITG) algorithm which alter-
nately seeks estimates as local optimal solutions in the tomographic space (1.1) and
a gravity space of network traffic flow x. Our algorithm, described in Section 2, is
based on a single snapshot of the link data and does not require the full knowledge
of the total inbound and outbound flow for all individual edge nodes. The idea is
to use the gravity space, instead of the specific simple gravity solution (1.2), to reg-
ularize the network tomography problem (1.1). In Section 3, we present the results
of a real-data experiment to demonstrate that the ITG method is competitive com-
pared with other tomogravity algorithms when the complete link data is available
and robust when a moderate amount of link data is missing.
2. An iterative tomogravity algorithm
In a general network tomographic model, the observed link data, as a sub-vector
y∗
of the vector y in (1.1), satisfies
y∗
= A∗
x,
(2.1)
where the matrix A∗
= (a∗
ij) is composed of the rows of the routing matrix A in
(1.1) corresponding to the observed links, and x is the SD traffic flow as in (1.1).
Let J be the total number of SD-pairs of concern. For the observation y∗
, the
tomographic space of probability vectors is
T ∗
=
f ∈ I
RJ
: y∗
∝ A∗
f, f ≥ 0, 1T
f = 1
,
(2.2)
where 1 is the vector composed of 1’s and vT
denotes the transpose of a vector v.
Here and in the sequel, inequalities are applied to all components of vectors.
In the literature, different types of flow and load are often specifically denoted.
Let y(net)
be the link loads of the inner network, y(edge)
the loads on the links be-
tween the edge nodes and inner network, y(self)
the load on the links from the edge
nodes to themselves, x(net)
the traffic flow between distinct edge nodes (necessarily
through the inner network), and x(self)
the flow of the edge nodes to themselves.
Since x(self)
does not go through the inner network and the flow from an edge node
to itself is the same as the load on the corresponding self-link, the tomographic
model can be written as
y =
y(net)
y(edge)
y(self)
=
A(net)
0
A(edge)
0
0 I(self)
x(net)
x(self)
= Ax,
(2.3)
with I(self)
being the identity matrix giving y(self)
= x(self)
, provided that the inner
network does not generate traffic. This is a special case of Vardi’s [10] tomographic](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-26-320.jpg)
![Iterative tomogravity algorithm 15
model (1.1) describing decompositions of the SD traffic x and link load y, but (1.1)
can be also viewed as y(net)
= A(net)
x(net)
. In this paper, the observed y∗
in (2.1)
is a general sub-vector of the y in (2.3) to allow partial observation of y(edge)
and
networks without y(self)
and x(self)
.
Suppose throughout the sequel that the list of the SD-pairs (sj, dj), i = 1, . . . , J,
forms a product set composed of all the pairings from a set S of source nodes to a
set D of destination nodes (D = S allowed), so that J = |S||D|, where |C| is the
size of a set C. This gives a one-to-one mapping between I
RJ
and the space of all
|S| × |D| matrices:
v = (v1, . . . , vJ )T
∼ (vsd)|S|×|D|, vj = vsj dj .
In this notation, the gravity space of probability vectors is
G =
g ∈ I
RJ
: g ∼ (gsd)|S|×|D| = p qT
, g ≥ 0, 1T
g = 1
,
(2.4)
i.e. gsd = psqd or matrices of rank 1, where p ∈ I
R|S|
and q ∈ I
R|D|
.
Zhang et al. [11] proposed (1.2) as the simple gravity algorithm and (1.4) as
the simple tomogravity algorithm. Zhang et al. [12] proposed (1.5) as the entropy-
regularized tomogravity algorithm. Their basic ideas can be summarized as follows:
(i) The gravity model gives a rough approximation of the SD flow; (ii) When the
simple gravity solution (1.2) is available, it can be used to regularize Vardi’s tomo-
graphic model (1.1). Motivated by their work, we propose the following algorithm
which provides estimates of the SD flow x in (2.1).
Iterative tomogravity algorithm (ITG):
Initialization: g = 1/J
(2.5)
Iteration: f(new)
= arg min
K(f, g(old)
) : f ∈ T ∗
(2.6)
g(new)
= arg min
K(f(new)
, g) : g ∈ G
(2.7)
Finalization:
N =
1T
y∗
1T
A∗
f( fin)
(2.8)
x =
Nf( fin)
(2.9)
where K(f, g) is the Kullback-Leibler information defined as
(2.10) K(f, g) =
J
j=1
fj log
fj
gj
.
As mentioned earlier, our basic idea is to use the gravity space (2.4), instead of
the simple gravity solution (1.2), to regularize the tomographic model (2.2). A main
advantage of this approach is that it does not require the knowledge of the simple
gravity solution or equivalently, the complete observation of loads on all edge links.
Numerical results in Section 3 demonstrate that when the complete link data y
is observed, the ITG (2.9) and the entropy-regularized tomogravity (1.5) perform
comparably in terms of estimation error, and they both outperform the simple
gravity (1.2) and tomogravity (1.4). Moreover, the ITG without using the knowledge
of the “access” or “peering” status of links has similar performance compared with
the generalized tomogravity method [11] which requires such knowledge. We note
that the ITG method does not need a tuning parameter as (1.5) does.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-27-320.jpg)
![16 J Fang, Y. Vardi and C.-H. Zhang
A main difference between ITG (2.9) and the simple tomogravity (1.4) is that the
simple gravity solution
x in (1.2) is not explicitly used in ITG, since g is treated
as an unknown in the ITG algorithm. However, the information in the observed
portions of y(edge)
and y(self)
is still utilized in the ITG iterations through the
tomographic space (2.2), instead of directly computing
x from y(edge)
and y(self)
as in (1.3). If the simple gravity
x (or an approximation of it if
x is not fully
available) is used as the initialization for ITG, the simple tomogravity solution is
the result of a single ITG iteration. We may also treat g =
x/N as an unknown in
(1.5), cf. Section 4, but then a tuning parameter is still required.
We use the relaxation algorithm of Krupp (1979) to compute (2.6) of the ITG,
while (2.7) is explicit with
g
(new)
sd =
d
f
(new)
sd
s
f
(new)
sd
as in (1.3). Here is a full description of the relaxation algorithm. Let g(old)
=
(g
(old)
1 , . . . , g
(old)
J )T
be a given probability vector. The problem is to minimize
K
f, g(old)
under the linear constraints in (2.2). Since y∗
i = 0 implies fij = 0
for all j with a∗
ij = 1 and thus reduces the optimization problem to a subset of j,
we assume y∗
= (y∗
1, . . . , y∗
r ) 0 where r is the total number of links with observed
load. Define
hij =
a∗
ij/y∗
i − a∗
rj/y∗
r , i = 1, . . . , r − 1,
1, i = r.
The linear constraints A∗
f = y∗
and 1T
f = 1 for the tomographic space (2.2) can
be written as Hf = (0T
, 1)T
, where H = (hij). Krupp’s [5] relaxation algorithm
maximizes
vr −
J
j=1
g
(old)
j exp
r
i=1
hijvi − 1
(2.11)
over all vectors v = (v1, . . . , vr)T
and then set
f
(new)
j = g
(old)
j exp
r
i=1
hijvi − 1
.
(2.12)
As (2.11) is concave in v, its optimization is done by the Newton-Raphson method
for individual components vi, cycling through i = 1, . . . , r. Since hr,j = 1 for all j,
f(new)
in (2.12) is properly normalized.
The iteration steps (2.6) and (2.7) are both monotone in K(f, g), so that the ITG
algorithm reaches a local minimum of the Kullback-Leibler information between the
tomographic (2.2) and gravity (2.4) spaces. However, since K(f, g) is not convex
jointly in (f, g) with g in the gravity space, ITG is not guaranteed to converge to
a global minimum.
3. An example
We conduct numerical experiments with data collected over the Abilene Network
(an Internet2 high-performance backbone network in United States) illustrated in](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-28-320.jpg)
![Iterative tomogravity algorithm 17
Fig 1. Abline Network.
Figure 1, with 12 nodes, 144 total traffic pairs (132 SD pairs and 12 self pairs),
30 inner links, and 24 edge links. We collect the full 12 × 12 SD traffic matrices in
5 min intervals for consecutive 19 weeks in 2004. We randomly pick four different
periods of 3 days and use the data in these four time periods. We call these four raw
datasets as X1, X2, X3, and X4. It turns out that the four datasets give different
traffic patterns as the time periods cover different days of the week, cf. Figure 2.
For each dataset and each hour, we compute x as the hourly total SD flow and
y = Ax with a fixed routing matrix A used in the Abilene data.
We compare four procedures using the complete data y as y∗
: the ITG (2.9), the
simple tomogravity (STG) in (1.4), the generalized tomogravity (GTG) of Zhang et
al. [11] utilizing the extra information of “access” or “peering” status of links, and
the entropy regularized tomogravity (ERTG) in (1.5). Since the traffic flow for self
pairs (s = d) is directly observable as the load on the self links, the ITG and STG
estimate these components of x without error. Thus, we measure the performance
of all estimators by the relative total error for non-self SD pairs
s=d
xsd − xsd
s=d
xsd,
(3.1)
where xsd is the flow from source s to destination d. We compute the relative total
error for (1.5) with various values of the tuning parameter φ and found that the
Table 1
Average of relative total errors for 288 different hours (4 different 3-day periods) based on
complete link data. The best tuning parameter is used for the ERTG, while extra
information is used for the GTG.
Method risk
Iterative Tomogravity (ITG) 0.3001
Entropy Regularized (ERTG) 0.2995
Simple Tomogravity (STG) 0.3139
Generalized Tomogravity (GTG) 0.3026](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-29-320.jpg)
![20 J Fang, Y. Vardi and C.-H. Zhang
Fig 6. Compare of the error rate using different models, dataset X4.
performance of (1.5) is near the best in a wide neighborhood of φ = 10−3
= 0.001.
This confirms the results of Zhang et al. [12]. Thus, φ = 10−3
= 0.001 is used for
(1.5) in our experiment. We plot the relative total error (3.1) against hour for the
four datasets in Figures 3, 4, 5, and 6. We tabulate the average relative error in
Table 1. From the results of the experiments, we observed that the performance of
the proposed ITG is comparable to the ERTG with the best choice of the tuning
parameter and the GTG based on extra information, while all three outperform the
STG.
We also exam the relative errors for different SD pairs as functions of the total
traffic flow for the SD pairs. We compute the relative total error over 3-day time
periods
t∗
t=1
x
(t)
sd − x
(t)
sd
t∗
t=1
x
(t)
sd
(3.2)
for fixed SD pairs in individual datasets, where t indicates time points with t∗
= 72.
We group the values of (3.2) for SD pairs in all datasets according to the total flow
t∗
t=1 x
(t)
sd with the grid {0, 1/4, 1/2, 3/4, 1, 1.5, 2, 2.5, 3, 4, 5, 7} in the unit of 1010
packets, and tabulate in Table 2 the average of (3.2) within groups. From Table
2, we observe that the estimation error is essentially a decreasing function of the
amount of traffic for individual SD pairs.
Finally, we check the robustness of the ITG (2.9) with missing link data (i.e. y∗
is a proper sub-vector of y). We focus on the case of missing data in edge links as
the ITG is the only procedure among the four that do not require observations for
all edge links. Let k be the number of edge links with missing data. We use only](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-32-320.jpg)
![Iterative tomogravity algorithm 21
Table 2
Relative total errors over 72 hours for fixed SD pairs and 3-day periods, grouped according to
the total flow. The relative total errors are decreasing functions of the flow for all 4 procedures.
Flow # in ITG ERTG STG GTG
Level Group
0 – 0.25 215 4.4799 5.8725 4.5833 5.3545
0.25 – 0.5 100 0.4320 0.4279 0.4548 0.4158
0.5 – 0.75 73 0.3457 0.3449 0.3666 0.3467
0.75 – 1 30 0.2997 0.2992 0.3379 0.2505
1 – 1.5 46 0.2286 0.2305 0.2402 0.2588
1.5 – 2 25 0.2878 0.2859 0.2934 0.3089
2 – 2.5 18 0.1836 0.1828 0.1802 0.2080
2.5 – 3 6 0.1583 0.1576 0.1689 0.1207
3 – 4 7 0.1143 0.1126 0.1335 0.1261
4 – 5 6 0.1456 0.1448 0.1514 0.1373
5 – 7 2 0.0887 0.0938 0.0767 0.0882
data for the first day in dataset X1 and compute the average of the relative total
error for 10 random missing patterns for each given k. We plot this average against
k in Figure 7. From Figure 7, we find that the performance of the ITG method is
robust against small or moderate amount of missing link data (up to 5 out of 24
edge links).
4. Discussion
We consider the estimation of SD traffic flow in a network based on observations of
a snapshot of traffic loads on links. Based on the ideas of Vardi [10] and Zhang et
al. [11, 12], we propose an iterative tomogravity method which allows incomplete
observation of the link data. Our main idea is to use the gravity space (2.4), instead
of the simple gravity solution (1.2), to regularize Vardi’s [10] tomographic model
(1.1). A numerical study with a real-life dataset demonstrates that the proposed
method has similar performance compared with the methods proposed in [11, 12]
which demand complete observation of the link data. We discuss below a number
of related issues.
There are two other possible ways of using the gravity space (2.4) to regularize
(1.1) that we do not explore in this paper. The first is to use the ITG (2.9) instead
of the simple gravity (1.2) in the penalty function in (1.5), resulting in
arg min
x
y∗
− A∗
x2
+ φ
N2
K
x/
N, g(fin)
/
N
(4.1)
with the
N in (2.8). The second is to alternate between the optimization in the
gravity space and entropy-regularized solution, i.e. to replace (2.6) with
N(new)
=
1T
y∗
1T
A∗
g(old)
(4.2)
f(new)
= arg min
y∗
− N(new)
A∗
f2
+φ{N(new)
}2
K
f, g(old)
: f ∈ T ∗
.
(4.3)
A small numerical study seems to indicate that there is little difference between
(4.1) and the ITG.
The proposed ITG (2.9) implicitly assumes that the measurement error in the
tomographic model (2.1) is of smaller order than the bias representing the Kullback-
Leibler distance K(x/N, G∗
) between x/N and the gravity space (2.4). This seems](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-33-320.jpg)
![22 J Fang, Y. Vardi and C.-H. Zhang
Fig 7. Relative total errors of the ITG versus the number of edge links with missing data. Average
over 10 random missing patterns is used for each point in the plot. The ITG is robust against
small or moderate amount of missing link data.
to be the case in our real-data experiments since ITG significantly improves upon
the simple tomogravity (1.4) by formally reducing K(x/N,
x/N) to K(x/N, G∗
).
In cases where the measurement error in the tomographic model is potentially of
larger order than K(x/N, G∗
) [or K(x/N,
x/N)] it would make sense to replace
(2.6) by (4.2) and (4.3) in ITG [or to use (1.5)] with a proper tuning parameter φ.
A possibility to further reduce the bias is to consider the mixed gravity model
Fmix =
f : f =
k∗
k=1
πkf(k)
, f(k)
∈ G
.
(4.4)
For example, we may compute a regularized mixed tomogravity solution
arg min
y − N
k∗
k=1
πkf(k)
+ N2
k∗
k=1
φkK(f(k)
, g(k)
)
(4.5)
by alternately optimizing over g(k)
∈ G, f(k)
, k = 1, . . . , k∗
and the mixing vector
(π1, . . . , πk∗ )T
.
It seems that for a network with a fixed routing protocol, the ITG estimate
x
in (2.9) is a continuous map of y∗
, so that
x − Ex is asymptotically normal when
y∗
− EA∗
x is asymptotically normal with Ex/N ∈ G, as N → ∞. Our simulation
study in a small artificial network has demonstrated the validity of this asymptotic
normality theorem for moderate sample sizes.
Estimation of traffic matrix based on link-load data alone is difficult as the
estimation error is typically above 20%. More accurate results can be obtained if](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-34-320.jpg)
![Iterative tomogravity algorithm 23
additional information can be extracted from packets passing through routers. See
for example Zhao, Kumar, Wang and Xu [13].
References
[1] Cao, J., Davis, D., Vander Weil, S. and Yu, B. (2000). Time-varying
network tomography: Router link data J. Amer. Statist. Assoc. 95 1063–1075.
[2] Cao, J., Vander Wiel, S., Yu, B. and Zhu, Z. (2000). A scalable method
for estimating network traffic matrices. Technical report, Bell Labs.
[3] Castro, R. Coates, M., Liang, G., Nowak, R. and Yu, B. (2004). Net-
work tomography: Recent developments. Statist. Sci. 19 499–517.
[4] Coates, M. and Nowak, R. (2002). Sequential Monte Carlo inference of
internal delays in nonstationary communication networks. IEEE Trans. Signal
Process. 50 366–376.
[5] Krupp, R. S. (1979). Properties of Kruithof’s projection method. The Bell
System Technical J. 58 517–538.
[6] Liang, G. and Yu, B. (2003). Maximum pseudo-likelihood estimation in net-
work tomography. IEEE Trans. Signal Process. 51 243–253.
[7] Liang, G., Taft, N. and Yu, B. (2006). A fast lightweight approach to origin-
destination IP traffic estimation using partial measurements. Special Issue of
IEEE-IT and ACM Networks on Data Networks, January 2006.
[8] Medina, A., Taft, N., Salamatian, K., Bhattacharyya, S. and Diot,
C. (2002). Traffic matrix estimation: Existing techniques compared and new
directions. SIGCOMM, Pittsburgh, Aug. 2002.
[9] Vanderbai, R. J. and Iannone, J. (1994). An EM approach to OD matrix
estimation. Technical Report SOR 94-04, Princeton Univ.
[10] Vardi, Y. (1996). Network tomography: Estimating source-destination traffic
intensities from link data. J. Amer. Statist. Assoc. 91 365–377.
[11] Zhang, Y., Roughan, M., Duffield, N. and Greenberg, A. (2003). Fast
accurate computation of large-scale IP traffic matrices from link loads. In ACM
SIGMETRICS, San Diego, USA, June 2003.
[12] Zhang, Y., Roughan, M., Lund, C. and Donoho, D. (2003). An
information-theoretic approach to traffic matrix estimation. In ACM SIG-
COMM, Karlsruhe, Germany, August 2003.
[13] Zhao, Q., Kumar, A., Wang, J. and Xu, J. (2005). Data streaming algo-
rithms for accurate and efficient measurement of traffic and flow matrices. ACM
SIGMETRICS, Banff, Canada, June 2005.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-35-320.jpg)
![Inverse problems in network tomography 25
Fig 1. Examples of tree network topologies. A binary two-layer tree is shown on the left panel and
a general four-layer tree on the right panel. The path lengths from the root to nodes belonging to
the same layer are the same.
Example 1. Consider the two-layer binary tree on the left panel of Figure 1, and
suppose the Xk are binary with P(Xk = 1) = αk, k = 1, 2, 3 for the three links.
Further, the root node sends transmissions to the receiver nodes one at a time.
Take f(a, b) = ab. Then, the observed data are Y2j = X1jX2j for transmission j
and Y3m = X1mX3m for transmission m. They are independent Bernoulli with prob-
abilities α1α2 and α1α3, respectively. Suppose we send M transmissions to receiver
node 2 and N transmissions to receiver node 3. Let M1 and N1 be the respective
number of “ones”. Then, M1 and N1 are independent binomial random variables
with success probabilities α1α2 and α1α3. From these data, we can estimate only
α1α2 and α1α3. The individual link-level parameters α1, α2 and α3 cannot be fully
recovered.
Example 2. Take the same two-layer binary tree with binary outcomes with
f(a, b) = ab as above. But now the root node sends transmissions to receiver nodes
2 and 3 simultaneously. In other words, the m-th transmission generates random
variables X1m, X2m and X3m on all of the links. We observe Y2m = X1mX2m
and Y3m = X1mX3m. The distinction from Example 1 is that the X1m is com-
mon to both Y2m and Y3m. Now, each transmission has 4 possible outcomes:
(1, 1), (1, 0), (0, 1), (0, 0) depending on whether the transmission reaches none,
one, or both of the receiver nodes. If we send N such transmissions to nodes 2 and
3 simultaneously, the result is a multinomial experiment with probabilities α1α2,
α1(1 − α2), (1 − α1)α2, and (1 − α1)(1 − α2) corresponding to the four outcomes.
Let N(i, j) denote the number of events with outcome (i, j). Then, E[N(1, 1)] =
α1α2α3, E[N(1, 1) + N(1, 0)] = α1α2, and E[N(1, 1) + N(0, 1)] = α1α3. It is easy
to see that we can estimate all the three link-level parameters from these measure-
ments. Thus, the data transmission scheme plays an important role in this type of](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-37-320.jpg)
![26 E. Lawrence, G. Michailidis and V. N. Nair
inverse problems.
Example 3. Again we have a two-layer binary tree but now f(a, b) = a + b. Then
Y2 = X1 + X2 and Y3 = X1 + X3. Let Fk be the distribution of the link-level
random variables Xk ∈ R, for k = 1, 2, 3. Assume, as in Example 2, that the
root node sends transmissions simultaneously to both receivers. In this case, even
with simultaneous transmission to both receivers, the link-level parameters are not
always identifiable. Just take Xk to be independent Normal(µk, 1), k = 1, 2, 3. Then
Y2 and Y3 are bivariate normal with mean µ1 + µ2 and µ1 + µ3, variance 2 and
correlation 1. One can see that the individual µk cannot be recovered from the joint
distribution of Y2 and Y3. Additional assumptions on the distribution are needed
in order to solve the inverse problem. We will revisit this issue.
Example 4. Consider now the more general tree on the right panel of Figure 1.
Again, we send transmissions to all of the receiver nodes simultaneously. If the ran-
dom variables are binary and f(x1, . . . , xp) =
p
j=1 xj, all the link-level parameters
are identifiable. The same is true for a general Xk with f(x1, . . . , xp) =
p
j=1 xj
under suitable conditions on the distribution of the Xk (as discussed later in the
paper). However, it may be “expensive” to send transmissions to all receiver nodes
simultaneously. Instead, can we schedule transmissions to some judicious subsets of
the receiver nodes at a time and combine the information appropriately to estimate
all the link-level parameters? It is clear from Example 1 that it is not enough to
send transmissions to one receiver node at a time. How should the transmission
scheme be designed in order to estimate all the parameters? Are there some “good”
schemes (according to some appropriate criteria)?
These examples are simple instances of issues that arise in the context of ana-
lyzing computer and communications networks and are collectively referred to as
active network tomography. In the next section, we will describe the network ap-
plication and the need for estimating quality-of-service (QoS) parameters such as
loss rates and delays. Section 3 provides an overview of recent results in the lit-
erature on the design of transmission experiments and inference for loss rates and
discrete delay distributions. A real application on data collected from the campus
network at the University of North Carolina, Chapel Hill is used to illustrate some
of the results. Section 4 develops some new results on parametric inference for delay
distributions.
2. Active network tomography
The area of network tomography originated with the pioneering work of Vardi
[14] where the term was first introduced. His work dealt with another type of in-
verse problem relating to origin-destination (OD) traffic matrix estimation. The
OD information is important in network management, capacity planning, and pro-
visioning. In this problem, one is interested in estimating the intensities of traffic
flowing between the origin-destination pairs in the network. However, we cannot
collect these data directly; rather, one places equipment at the individual nodes
(routers/switches) and collects aggregate data on all traffic flowing through the
nodes i ∈ V. The goal is to recover distributions of origin-destination traffic be-
tween all pairs of nodes in the network. There has been considerable work in this
area, and a summary of the developments can be found in [3].
Active network tomography, on the other hand, is concerned with the “opposite”
problem of estimating link-level information from end-to-end data. One sends test](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-38-320.jpg)
![28 E. Lawrence, G. Michailidis and V. N. Nair
3. Literature review of loss and discrete delay inference
Most of the results in the literature on active tomography have been developed
under the assumption that the loss rates and delay distributions are temporally
homogeneous and are independent across links. We will also use this framework.
The assumption of temporal homogeneity is reasonable as the probing experiments
are done within the order of minutes. The assumption of independence across links
is less likely to hold. However, the nature of the dependence will vary from network
to network, and it is difficult to obtain general results.
3.1. Design of probing experiments
We noted in Example 1 that the link-level parameters are not identifiable under
the unicast transmission scheme (sending probes to one receiver at a time). The
multicast scheme, which sends packers to all the receivers in the network simultane-
ously, addresses this problem for loss rates and, under some additional conditions,
for delay distributions as well.
However, this scheme has a number of drawbacks. It creates more traffic than
necessary for estimating the link-level parameters. Also, the data generated are very
high-dimensional. For example, in a binary symmetric tree with L layers, there are
R = 2L
− 1 receiver nodes. A multicast scheme for measuring loss rates results in
a multinomial experiment with 2R
possible outcomes. This is a large number even
for moderately sized trees. The most important drawback, however, is that it is
inflexible and does not allow investigation of subnetworks using different intensities
and at different times. In practice, one may want to probe sensitive parts of the
network as lightly as necessary to avoid disturbance. So there is a need for more
flexible probing experiments. As pointed out in Example 4, this raises interesting
issues on how to design the probing experiments.
A class of flexible probing experiments, called flexicast experiments, were in-
troduced and studied in Xi et al. [17] and Lawrence et al. [8]. This consists of a
combination of schemes for different values of k with each scheme aimed at study-
ing a subnetwork. However, each of the scheme by itself will not necessarily allow
us to estimate the link-level parameters of that subnetwork. The data have to be
combined across the various k-cast schemes to estimate the link-level parameters.
To illustrate the ideas, consider the network on the right panel in Figure 1.
The multicast scheme sends probes simultaneously to {2, 3, 6, 8, 9, 10, 11, 12, 13, 14,
15}. Two possible flexicast experiments are:
(1) {2, 3, 6, 12, 13, 14, 8, 15, 9, 10, 11}
and
(2) {2, 3, 6, 12, 13, 14, 15, 8, 9, 10, 11}.
The former consists of only bicast (two receiver nodes at a time) and unicast
schemes. Intuitively, the latter scheme appears to more “efficient” but we will see
shortly that it does not allow one to estimate all the link-level parameters.
A full multicast scheme for this tree will result in 11-tuples or 11-dimensional
data. The first flexicast experiment using pairs and singletons can cover the whole
tree with five pairs and one singleton. The resulting data are considerably less
complex in terms of processing and computations for inference. This advantage is
particularly important for trees with many layers.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-40-320.jpg)
![Inverse problems in network tomography 29
Of course, not all flexicast experiments will permit estimation of the link-level
parameters. To discuss the technical issues associated with the identifiability prob-
lem, consider first the notion of a splitting node. For a k-cast scheme, an internal
node is a splitting node if the scheme splits at that node. For example, for the tree
on the right panel of Figure 1, the bicast scheme {6, 12} splits at node 4. Xi et
al. [17] showed that the following conditions are necessary and sufficient for iden-
tifiability of link-level loss rates: (a) all receiver nodes are covered; and (b) every
internal node in the tree is a splitting node for some k-cast scheme in the flexicast
experiment. Lawrence et al. [8] studied the delay problem and showed that the
same conditions are also necessary and sufficient for estimating delay distributions
provided the distributions are discrete. The case where the delay distributions are
not discrete is discussed in the next section.
Consider again the flexicast schemes in equations (1) and (2) for the tree on the
right panel in Figure 1. The first one based on a collection of bicast and unicast
schemes satisfies the conditions. For the second one, none of the k-cast schemes
split at node 4.
There are many flexicast experiments that satisfy the identifiability requirements,
and the choice among these has to be based on other criteria. Experiments based on
just bicast and unicasts have minimal data complexity – just 1- and 2-dimensional
outcomes. However, these provide information on just first and second-order de-
pendencies and will be less efficient (in a statistical sense) to k-cast schemes with
higher values of k. In particular, the full mulitcast scheme will be most efficient in
this sense. So the overall choice of the flexicast experiment has to be a compromise
between statistical efficiency and flexibility including the ability to adapt over time
to accommodate changes in network conditions.
3.2. Inference for loss rates
Inference for loss rates was first studied in Cáceres et al. [2] for the multicast scheme.
A recent, up-to-date list of references can be found in Xi et al. [17] who developed
MLEs based on the EM algorithm for flexicast experiments. We provide next a brief
review of these results.
Each k-cast scheme in a flexicast experiment is a k-dimensional multinomial ex-
periment. Specifically, each outcome is of the form {Zr1 , . . . , Zrk
} where Zrj = 1 or
0 depending on whether the probe reached receiver node rj or not. Let N(r1,...,rk)
denote the number of outcomes corresponding to this event, and let γ(r1,...,rk) be the
probability of this event. Then the log-likelihood for the k-cast scheme is propor-
tional to γ(r1,...,rk) log(N(r1,...,rk)). The overall log-likelihood is just the sum of the
log-likelihoods for these individual experiments. However, the γ(r1,...,rk) are compli-
cated functions of αk, the link-level loss rates, so one has to use numerical methods
to obtain the MLEs.
The EM algorithm is a natural approach for computing the MLEs and has been
used extensively in network tomography applications (see [3, 5, 16]). The structure
of the EM-algorithm for general flexicast experiments was developed in Xi et al.
[17]. While the E-step can be complex for arbitrary collections of k-cast schemes, it
simplifies for flexicast experiments comprised of bicast and unicast schemes as seen
below.
Let sb be the splitting node for bicast pair b = ib, jb. Then, π(0, sb), π(sb, ib)
and π(sb, jb), the three path probabilities for this bicast pair are products of the
αk. Starting with an initial value
α(0)
let
α(k)
be the value after the k-th iteration.
Then, we can write the (k + 1)-th iteration of the E-step as follows:](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-41-320.jpg)
![30 E. Lawrence, G. Michailidis and V. N. Nair
E-step:
1. For each bicast pair:
(a) Use
α(k)
to obtain the updated path probabilities π(k)
(0, sb), π(k)
(sb, ib),
π(k)
(sb, jb) and γ
b (k)
00 .
(b) For each node ∈ P(0, sb) ∪ P(sb, ib) ∪ P(sb, jb), compute V
(k+1)
,b = E
α(k) [V|
Nb], where Nb = {Nb
00, Nb
01, Nb
10, Nb
11} are the collected counts of the four
possible outcomes, as follows.
For node ∈ P(0, sb),
V
(k+1)
,b = Nb
− Nb
00
1 − α
(k)
γ
b (k)
00
.
For link ∈ P(sb, ib),
V
(k+1)
,b = Nb
− Nb
01 ×
1 − α
(k)
1 − π(k)(sb, ib)
− Nb
00
(1 − α
(k)
)(1 − π(k)
(0, jb))
γ
b (k)
00
.
For link ∈ P(sb, jb),
V
(k+1)
,b = Nb
− Nb
10 ×
1 − α
(k)
1 − π(k)(sb, jb)
− Nb
00
(1 − α
(k)
)(1 − π(k)
(0, ib))
γ
b (k)
00
.
2. Unicast schemes: Let node ∈ P(0, u) for a unicast transmission to receiver
node u, and compute
V
(k+1)
,u = Nu
− Nu
0 ×
1 − α
(k)
1 − π(k)(0, u)
M-step: The (k + 1)-th update for the M-step is simply
α
(k+1)
=
b∈B
V
(k+1)
,b +
u∈U
V
(k+1)
,u
b∈B
Nb +
u∈U
Nu
where B is the set of bicast pairs that includes the node in its path and U is
the set of all unicast schemes that includes node in its path.
In our experience, the EM algorithm works reasonably well for small to moderate
networks when used with a flexicast experiment that consists of a collection of bi-
cast and unicast schemes. For large networks, however, it becomes computationally
intractable. In on-going work, we are developing a class of fast estimation meth-
ods based on least-squares methods and are studying their application to on-line
monitoring of network performance.
3.3. Inference for discrete delay distributions
For the delay problem, let Xk denote the (unobservable) delay on link k, and let
the cumulative delay accumulated from the root node to the receiver node r be
Yr =
k∈P(0,r) Xk. Here P(0, r) denotes the path from node 0 to node r. The
observed data are end-to-end delays consisting of Yr for all the receiver nodes.
Most of the papers on delay inference assume a discrete delay distribution. Specif-
ically, if q denotes the universal bin size, Xk ∈ {0, q, 2q, . . . , bq} is the discretized](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-42-320.jpg)
![Inverse problems in network tomography 31
delay on link k and bq is the maximum delay. Let αk(i) = P{Xk = iq}. The in-
ference problem then reduces to estimating the parameters αk(i) for k ∈ E and i
in {0, 1, . . . , b} using the end-to-end data Yr. Lo Presti et al. [10] developed a fast,
heuristic algorithm for estimating the link delays. Liang and Yu [9] developed a
pseudo-likelihood estimation method. Nonparametric maximum likelihood estima-
tion under the above setting was investigated in Tsang et al. [13] and Lawrence
et al. [8]. Shih and Hero [12] examined inference under mixture models. See also
Zhang [18] for a more general discussion of the deconvolution problem.
We discuss nonparametric MLE with discrete delays in more detail. Let
αk =
[αk(0), αk(1), . . . αk(b)]
and let
α = [
α
0,
α
1, . . . ,
α
|E|]
. The observed end-to-end
measurements consist of the number of times each possible outcome
y was observed
from the set of outcomes Yc
for a given scheme c. Let Nc
y denote these counts.
These are distributed as multinomial random variables with corresponding path-
level probabilities γc(
y;
α). So the log-likelihood is given by
l(
α; Y) =
c∈C
y∈Yc
Nc
y log[γc(
y;
α)].
This cannot be maximized easily, and one has to resort to numerical methods.
Again, the EM algorithm is a reasonable technique for computing the MLEs.
See [7] for multicast schemes and [8] for inference with flexicast experiments. How-
ever, the complexity of the EM algorithm, in particular computing conditional
expectations of the internal link delays for each bin, is prohibitive for all but fairly
small-sized networks. To deal with larger networks, [8] developed a grafting method
which fits “local” EMs to the subtrees defined by the k-cast schemes and then com-
bines the estimates through a fixed point algorithm. This hybrid algorithm is fast
and has reasonable statistical efficiency compared to the full MLE.
For bicast schemes, the resulting algorithm has third-order polynomial complex-
ity, a substantial improvement over the full bicast MLE. The heuristic algorithm in
[10] is based on solving higher order polynomials and is much faster. However, it
uses only part of the data and is quite inefficient. The pseudo-likelihood method of
[9] uses only data from all pairs of probes in the multicast experiment. This is simi-
lar in spirit to a flexicast experiment comprised of only bicast schemes, although in
this setting the schemes would be independent. The computational performance of
the pseudo-likelihood method is faster than the MLE based on the full multicast.
It is comparable to doing a full EM based on data from all possible bicast schemes.
This will still not scale up well to very large trees as it includes all possible bicasts
which can involve a large number of schemes. Furthermore, using the full MLE
combining the results across all schemes is computationally intensive. The flexicast
experiments, on the other hand, are typically based on a much smaller number of
schemes (eve if one restricts attention to bicasts). Further, the grafting algorithm
is much faster for combining the results across the schemes.
3.4. Application to the UNC network
We use a real example to demonstrate how the results from active tomography
are used. The example deals with estimating the QoS of the campus network at
the University of North Carolina at Chapel Hill and assessing its capabilities for
Voice-Over-IP readiness.
This network has 15 endpoints which were organized into the tree shown in
Figure 2. Node 1 is the main campus router and it connects to the university gate-
way. Nodes 2, 3, and 9 are also large routers responsible for different portions of](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-43-320.jpg)
![34 E. Lawrence, G. Michailidis and V. N. Nair
Fig 5. Three-layer, binary tree
4.1. Identifiability
The basic issue for delay distributions is the one posed in Example 2 in the intro-
ductory section, viz., whether the parameters of a simple two-layer tree (left panel
of Figure 1) are estimable from probes sent simultaneously to both receivers. If this
holds, then the result extends readily to general flexicast experiments that satisfy
the conditions in Section 3.1 (using the arguments in [8]). We discuss the details
briefly. See also [4, 6] for a general discussion of identifiability issues.
We consider two cases:
Case 1: If pk 0 for all k, no additional assumptions on the distribution F(·)
are needed. All the link-level delay parameters (pk and θk) are identifiable using
flexicast experiments provided they satisfy the conditions in Section 3.1: a) every
receiver node is covered and b) every internal node is a splitting node for some
sub-experiment.
To see this, consider the two-layer tree on the left panel of Figure 1. Condition
on the subset of data with Y2,m = 0 and Y3,m 0 for probes m = 1, . . . , M.
Now, Y2,m = 0 implies that both of the internal links X1,m and X2,m had zero
delay, so Y3,m = X3,m. So we can use this subset of Y3,m to estimate F(x; θ3). A
similar argument can be used to estimate F(x; θ2) using the subset of Y3,m 0
and Y2,m = 0. Once these two distributions are estimated, we can easily estimate
F(x; θ1).
Case 2: If pk = 0, then we need additional assumptions on the delay distributions
F(x; θk). As we noted in Example 2, the means of the normal distributions are
not identifiable. If the moments of order two and higher depend on the first mo-
ment, they will provide additional information for estimating the parameters. One
such example is when the variance is a function of the mean (as is the case with
exponential, gamma, log-normal, and Weibull distributions).
Example 5. We consider here a more general situation with the three-layer bi-
nary, symmetric tree shown in Figure 5. Let the delay on link k be distributed
Gamma(αk, βk). Suppose we use the flexicast probing experiment {4, 5, 5, 6, 6,](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-46-320.jpg)
![36 E. Lawrence, G. Michailidis and V. N. Nair
simultaneously. The log-likelihood function is
l(
λ; Y) = N log(λ1) + N log(λ2) + N log(λ3) − N log(λ1 − λ2 − λ3)
(4)
− λ2
N
i=1
yi,2 − λ3
N
i=1
yi,3
+
N
i=1
log[1 − exp{−(λ1 − λ2 − λ3) min(yi,2, yi,3)}]
There is no analytic solution to maximize this equation over
λ, so one would have
to use an iterative technique, such as EM or Newton-Raphson, to find the MLEs
even in this simple case.
We examine the details for the EM-algorithm. The exponential distribution is a
member of the exponential family, so the (unobserved) sufficient statistics are the
total link-level delays
n
i=1 Xi,1,
n
i=1 Xi,2, and
n
i=1 Xi,3. Since Xi,2 = Yi,2 −Xi,1
and Xi,3 = Yi,3 − Xi,1, we need to compute only the conditional expected values
of
n
i=1 Xi,1 in the E-step. The conditional distribution [X1|Y2 = a, Y3 = b] has
density
(5) g(x) =
exp{−(λ1 − λ2 − λ3)x}
C(a, b; λ1, λ2, λ3)
, 0 x a ∧ b,
where a ∧ b = min(a, b) and the constant of proportionality is
C(a, b; λ1, λ2, λ3) =
1 − exp{−(λ1 − λ2 − λ3)(a ∧ b)}
λ1 − λ2 − λ3
.
Now
a∧b
0
xg(x)dx is an incomplete gamma function and one can compute the
expected value
n
i=1 E(Xi,1|Yi,2, Yi,3) as a ratio of the incomplete gamma function
and the constant C(a, b; λ1, λ2, λ3). Thus, the MLEs of the λk can be computed
without too much trouble in this simple two-layer binary case.
How well does this extend to more general cases? Suppose we have a three layer
binary tree (Figure 5), and we use bicast schemes 4, 5, 6, 7, and 5, 6. Consider
the scheme 4, 5 which splits at node 2. We can try and mimic the computations
for the two-layer tree above. However, we have to consider the combined path P(0,2)
whose delay is the sum of delays for links 1 and 2. The exponential distribution is
not closed under convolution, so the distribution is now more complex. The details
for more general trees will depend on the number of links involved before-and-after
the splitting node. The problem is even more complex for multicast schemes with
multiple splitting nodes. We see that the MLE computations are complicated even
for simple exponential distributions.
b) Gamma Distributions: Gamma distributions with same scale parameter are
closed under convolution, i.e., the path delays which are sums of link-level delays
are still gamma. Specifically, let Xk ∼ Gamma(αk, β) and independent across k for
k ∈ E. We start with the simple two-layer binary tree. Then, the likelihood function](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-48-320.jpg)
![Inverse problems in network tomography 37
of the observed data is
L(data) =
n
i=1
yi,2∧yi,3
0
f1(x)f2(yi,2 − x)f3(yi,3 − x)dx
=
n
i=1
yi,2∧yi,3
0
1
Γ(α1)
1
βα1
xα1−1
exp{−
x
β
}
×
1
Γ(α2)
1
βα2
(yi,2 − x)α2−1
exp{−
yi,2 − x
β
}
×
1
Γ(α3)
1
βα3
(yi,3 − x)α3−1
exp{−
yi,3 − x
β
}
dx
=
n
i=1
1
Γ(α1)Γ(α2)Γ(α3)
1
βα1+α2+α3
exp{−
1
β
(yi,2 + yi,3)}
×
yi,2∧yi,3
0
xα1−1
(yi,2 − x)α2−1
(yi,3 − x)α3−1
exp{
xi
β
}dx
.
As before, the MLEs will have to be obtained numerically.
Let us consider the details of the EM-algorithm. The Gamma distribution is
a member of the exponential family with sufficient statistics X and log(X). For
the two-layer tree, we need to compute just the conditional expectation of X1
and log(X1), the unknown delays on the first link. The conditional distribution
[X1|Y1 = a, Y2 = b] is now given by
(6) g(x) =
xα1−1
(a − x)α2−1
(b − x)α3−1
exp{x
β }
C(a, b; α1, α2, α3, β)
, 0 x a ∧ b,
where the proportionality constant is
C(a, b; α1, α2, α3, β) =
a∧b
0
xα1−1
(a − x)α2−1
(b − x)α3−1
exp{
x
β
}dx.
This can be used to compute E[X1|Y1, Y2] and E[log(X1)|Y1, Y2] numerically. Note
that
E[X1|Y1, Y2] =
C(Y1, Y2; α1 + 1, α2, α3, β)
C(Y1, Y2; α1, α2, α3, β)
.
How well does this extend to trees with more than two layers? It turns out that
the full MLE is still not feasible. However, a combination of “local” MLEs and a
grafting idea (along the lines of [8]) is feasible. Consider the 3-layer tree in Figure
5. Suppose we use a flexicast experiment with 3 bicasts 4, 5, 6, 7, and 5, 6.
The bicast scheme 4, 5 splits at node 2. So we can combine links 1 and 2 into
a single link and use the previous results for the two-layer tree to get estimates
for this subtree. Note that the delay distribution for the combined links 1 and 2 is
Γ(α1 +α2, β). So we can get “local” MLEs for α1 +α2, α4, α5 and β from the bicast
experiment 4, 5. Using a similar argument, we can get estimates for α1 + α3, α6,
α7 and β from the bicast scheme 6, 7 and estimates for α1, α2 + α5, α3 + α7 and
β from the bicast scheme 5, 6. Now we can use one of several methods to combine
these estimates to get a unique set of estimates for all of the αk and β. Possible
methods include ordinary or weighted LS.
We do not pursue this strategy here as the specifics work only for special cases.
The main message here is that it is not easy to compute the full MLE even in very
simple cases, and the problem becomes completely intractable as the size of the
tree and number of children in the links grow.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-49-320.jpg)
![38 E. Lawrence, G. Michailidis and V. N. Nair
4.3. Method-of-moments estimation
We discuss the use of method of moments which estimates the parameters by match-
ing the population moments to the sample moments using some appropriate loss
function. General losses are possible, but squared error loss leads to more tractable
optimization and the large-sample properties are easy to establish.
Let H = {1, . . . , m} be the index set of the probes used in the probing exper-
iment. Denote the observed data Yr(1), . . . , Yr(m) as Yr(H). Let Mj
i (H) be the
observed i-th moment for the j-th scheme based on the probes in H. Let Mj
i (θ) be
the functional form of the i-th moment from the j-th probing scheme. For example,
for the two-layer tree in Figure 1 with Gamma(αk, βk) distributions on each link,
we get the following relationships:
E(Y2) = α1β1 + α2β2,
E(Y3) = α1β1 + α3β3,
Cov(Y2, Y3) = α1β2
1,
E(Y2 − ν2)2
(Y3 − ν3) = 2α1β3
1,
Var(Y2) = α1β2
1 + α2β2
2,
Var(Y3) = α1β2
1 + α3β2
3.
We can now estimate the parameters by minimizing the squared error loss
Q(θ; M(H)) =
m
j=1
i
Mj
i (H) − Mj
i (θ)
2
.
This is a special case of the nonlinear least squares problem and can be solved using
iterative methods such as the Gauss-Newton procedure (see [1] for example). After
rewriting the loss function as a single sum over all the moments, we consider the
derivatives
(7)
∂Q(θ; M(H))
∂θj
= −2
i
[Mi(H) − Mi(θ)]
∂Mi(θ)
∂θj
.
These can be expressed in matrix form as
∂Q(θ; M(H))
∂θ
= D
[M(H) − M(θ)],
where Di,j = ∂Mi(θ)
∂θj
. The moments at the true value can be expanded using a Tay-
lor series expansion around an initial guess θ(0)
as M(θ0) ≈ M(θ(0)
)+D(θ0 −θ(0)
).
Computing the residuals and replacing the true value with the observed moments
gives an updating scheme based on solving a linear system. Thus at iteration q, we
have the following linear system.
M(H) − M(θ(q)
) = Dβ.
Solving this, we get the next iteration as θ(q+1)
= θ(q)
+ β̂.
In general, each iteration should be closer to the minimizer. However, there
can be situations where the step increases the sum of squares. To avoid this, we
recommend the modified Gauss-Newton in which the next iteration is given by
θ(q+1)
= θ(q)
+ rβ̂ where 0 r ≤ 1. This fraction can be chosen adaptively at each](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-50-320.jpg)
![Inverse problems in network tomography 41
4.5. Analysis using the NS-2 simulator
We now describe a study of the MOM estimators in a simulated network environ-
ment. The ns-2 package is a discrete event simulator that allows one to generate
network traffic and transmit packets using various network transmission protocols,
such as TCP, UDP, ([15]) over wired or wireless links. The simulator allows the
underlying link delays to exhibit both spatial and temporal dependence with corre-
lation between sister links (children of the same parent, i.e. 4, 5, and 6.) around .25
and autocorrelation about .2 on all of the links. Thus, we can study the performance
of the active tomography methods under more realistic scenarios.
We used the topology shown in Figure 8 with a multicast transmission scheme.
The capacity of all links was set to the same size (100 Mbits/sec), with 11 sources
(10 TCP and one UDP) generating background traffic. The UDP source sent 210
byte long packets at a rate of 64 kilobits per second with burst times exponentially
distributed with mean .5s, while the TCP sources sent 1,000 byte long packets
every .02s. The main difference between these two transmission protocols is that
UDP transmits packets at a constant rate while TCP sources linearly increase their
transmission rate to the set maximum and halve it every time a loss is recorded.
The length of the simulation was 300 seconds, with probe packets 40 bytes long
injected into the network every 10 milliseconds for a total of about 3,000. Finally,
the buffer size of the queue at each node (before packets are dropped and losses
recorded) was set to 50 packets.
We studied only the continuous component of the delay distribution, i.e., the
portion of the path-level data that contain zero or infinite delays was removed.
The traffic-generating scenario described above resulted in approximately uniform
waiting times in the queue (see Figure 9). This is somewhat unrealistic in real
network situations where traffic tends to be fairly bursty [11], but it provides a
simple scenario for our purposes. Estimating the unknown parameters for this model
is equivalent to estimating the maximum waiting time for a random packet.
Figure 10 shows quantile-quantile plots using simulated values from the fitted
distributions versus the observed ns-2 delays for both the links and paths. Specifi-
cally, we estimated the parameters for the uniform distributions using the moment
estimation procedure and then generated data based on these parameter values.
The fitted values were: b̂ = [.89, .79, .53, 1.10, 1.09, 1.13]. The estimation procedure
does quite well on all of the links except for the interior link 3. The algorithm seems
Fig 8. Portion of the UNC network used in the ns-2 simulaton scenario.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-53-320.jpg)
![42 E. Lawrence, G. Michailidis and V. N. Nair
Fig 9. Histogram of link delays for the ns-2 simulation.
to compensate for this under-estimation (about 40%) by slightly overestimating the
parameters for each of the descendants of link 3, as evidenced by the closely matched
quantiles for the end-to-end data. This error is probably the result of several fac-
tors. First, link 3 deviates the most from the uniform distribution with the last
bin in Figure 9 being too thin. Secondly, the algorithm appears to be moderately
affected by the violations of the independence assumptions, particularly the spatial
dependence among the children of link 3. This could likely be somewhat relieved
by using a larger sample size and accounting for the empty queue probabilities.
Nevertheless, the estimation performs well overall.
5. Summary
There are a number of other interesting problems that arise in active network to-
mography. There are usually multiple source nodes, which raises the issue of how to
optimally design flexicast experiments for the various sources. We have also assumed
that the logical topology of the tree is known. However, only partial knowledge of
the network topology is typically available, and one would be interested in using
the path level information to simultaneously discover the topology and estimate
the parameters of interest. The topology discovery problem is computationally dif-
ficult (NP-hard), but methods using a Bayesian formulation as well as those based
on clustering ideas have been proposed in the literature (see [3] and references
therein).
Active tomography techniques are useful for monitoring network quality of ser-
vice over time. However, this application requires that path measurements are col-
lected sequentially over time and appropriately combined. The probing intensity,
the type of control charts to be used for monitoring purposes, and the use of path-
vs link- information are topics of current research.](https://image.slidesharecdn.com/1311419-250530101107-4491add2/85/Complex-Datasets-And-Inverse-Problems-Tomography-Networks-And-Beyond-Regina-Liu-54-320.jpg)
Complex Datasets And Inverse Problems Tomography Networks And Beyond Regina Liu
- 1. Complex Datasets And Inverse Problems Tomography Networks And Beyond Regina Liu download https://ebookbell.com/product/complex-datasets-and-inverse- problems-tomography-networks-and-beyond-regina-liu-2622838 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Understanding Complex Datasets Data Mining With Matrix Decompositions 1st Edition David Skillicorn https://ebookbell.com/product/understanding-complex-datasets-data- mining-with-matrix-decompositions-1st-edition-david-skillicorn-897102 Biological Data Exploration With Python Pandas And Seaborn Clean Filter Reshape And Visualize Complex Biological Datasets Using The Scientific Python Stack Dr Martin Jones https://ebookbell.com/product/biological-data-exploration-with-python- pandas-and-seaborn-clean-filter-reshape-and-visualize-complex- biological-datasets-using-the-scientific-python-stack-dr-martin- jones-55211736 Data Science In Theory And Practice Techniques For Big Data Analytics And Complex Data Sets Maria C Mariani https://ebookbell.com/product/data-science-in-theory-and-practice- techniques-for-big-data-analytics-and-complex-data-sets-maria-c- mariani-52557058 Complex Systems Spanning Control And Computational Cybernetics Foundations Dedicated To Professor Georgi M Dimirovski On His Anniversary Peng Shi https://ebookbell.com/product/complex-systems-spanning-control-and- computational-cybernetics-foundations-dedicated-to-professor-georgi-m- dimirovski-on-his-anniversary-peng-shi-44888180
- 3. Complex Systems Smart Territories And Mobility Patricia Sajous https://ebookbell.com/product/complex-systems-smart-territories-and- mobility-patricia-sajous-46383262 Complex Variables Principles And Problem Sessions A K Kapoor https://ebookbell.com/product/complex-variables-principles-and- problem-sessions-a-k-kapoor-46469966 Complex Networkbased Global Value Chain Accounting System From The Perspective Of Econophysics Lizhi Xing https://ebookbell.com/product/complex-networkbased-global-value-chain- accounting-system-from-the-perspective-of-econophysics-lizhi- xing-46668040 Complexvalued Neural Networks Systems With Time Delay Stability Analysis And Antisynchronization Control Ziye Zhang https://ebookbell.com/product/complexvalued-neural-networks-systems- with-time-delay-stability-analysis-and-antisynchronization-control- ziye-zhang-47123284 Complex Analysis With Mathematica William T Shaw https://ebookbell.com/product/complex-analysis-with-mathematica- william-t-shaw-47167888
- 5. Institute of Mathematical Statistics LECTURE NOTES–MONOGRAPH SERIES Volume 54 Complex Datasets and Inverse Problems Tomography, Networks and Beyond Regina Liu, William Strawderman and Cun-Hui Zhang, Editors Institute of Mathematical Statistics Beachwood, Ohio, USA
- 6. Institute of Mathematical Statistics Lecture Notes–Monograph Series Series Editor: R. A. Vitale The production of the Institute of Mathematical Statistics Lecture Notes–Monograph Series is managed by the IMS Office: Jiayang Sun, Treasurer and Elyse Gustafson, Executive Director. Library of Congress Control Number: 2007924176 International Standard Book Number (13): 978-0-940600-70-6 International Standard Book Number (10): 0-940600-70-6 International Standard Serial Number: 0749-2170 Copyright c 2007 Institute of Mathematical Statistics All rights reserved Printed in Lithuania
- 7. Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Deconvolution by simulation Colin Mallows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 An iterative tomogravity algorithm for the estimation of network traffic Jiangang Fang, Yehuda Vardi and Cun-Hui Zhang . . . . . . . . . . . . . . . . . . . . 12 Statistical inverse problems in active network tomography Earl Lawrence, George Michailidis and Vijayan N. Nair . . . . . . . . . . . . . . . . . 24 Network tomography based on 1-D projections Aiyou Chen and Jin Cao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Using data network metrics, graphics, and topology to explore network characteristics A. Adhikari, L. Denby, J. M. Landwehr and J. Meloche . . . . . . . . . . . . . . . . . 62 A flexible Bayesian generalized linear model for dichotomous response data with an application to text categorization Susana Eyheramendy and David Madigan . . . . . . . . . . . . . . . . . . . . . . . . . 76 Estimating the proportion of differentially expressed genes in comparative DNA microarray experiments Javier Cabrera and Ching-Ray Yu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Functional analysis via extensions of the band depth Sara López-Pintado and Rebecka Jornsten . . . . . . . . . . . . . . . . . . . . . . . . . 103 A representative sampling plan for auditing health insurance claims Arthur Cohen and Joseph Naus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Confidence distribution (CD) – distribution estimator of a parameter Kesar Singh, Minge Xie and William E. Strawderman . . . . . . . . . . . . . . . . . . 132 Empirical Bayes methods for controlling the false discovery rate with dependent data Weihua Tang and Cun-Hui Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 A smoothing model for sample disclosure risk estimation Yosef Rinott and Natalie Shlomo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 A note on the U, V method of estimation Arthur Cohen and Harold Sackrowitz . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Local polynomial regression on unknown manifolds Peter J. Bickel and Bo Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Shape restricted regression with random Bernstein polynomials I-Shou Chang, Li-Chu Chien, Chao A. Hsiung, Chi-Chung Wen and Yuh-Jenn Wu . . 187 Non- and semi-parametric analysis of failure time data with missing failure indicators Irene Gijbels, Danyu Lin and Zhiliang Ying . . . . . . . . . . . . . . . . . . . . . . . . 203 iii
- 8. iv Contents Nonparametric estimation of a distribution function under biased sampling and censoring Micha Mandel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 Estimating a Polya frequency function2 Jayanta Kumar Pal, Michael Woodroofe and Mary Meyer . . . . . . . . . . . . . . . 239 A comparison of the accuracy of saddlepoint conditional cumulative distribution function approximations Juan Zhang and John E. Kolassa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 Multivariate medians and measure-symmetrization Richard A. Vitale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 Statistical thinking: From Tukey to Vardi and beyond Larry Shepp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
- 9. Preface This book is a collection of papers dedicated to the memory of Yehuda Vardi. Yehuda was the chair of the Department of Statistics of Rutgers University when he passed away unexpectedly on January 13, 2005. On October 21–22, 2005, some 150 leading scholars from many different fields, including statistics, telecommunications, biomedical engineering, bioinformatics, biostatistics and epidemiology, gathered at Rutgers in a conference in his honor. This conference was on “Complex Datasets and Inverse Problems: Tomography, Networks, and Beyond,” and was organized by the editors. The present collection includes research work presented at the conference, as well as contributions from Yehuda’s colleagues. The theme of the conference was networks and other important and emerging ar- eas of research involving incomplete data and statistical inverse problems. Networks are abundant around us: communication, computer, traffic, social and energy are just a few examples. As enormous amounts of network data are collected in this in- formation age, the field has attracted a great amount of attention from researchers in statistics and computer engineering as well as telecommunication providers and various government agencies. However, few statistical tools have been developed for analyzing network data as they are typically governed by time-varying and mutu- ally dependent communication protocols sitting on complicated graph-structured network topologies. Many prototypical applications in these and other important technologies can be viewed as statistical inverse problems with complex, massive, high-dimensional and possibly biased/incomplete data. This unifying theme of in- verse problems is particularly appropriate for a conference and volume dedicated to the memory of Yehuda. Indeed he made influential contributions to these fields, especially in medical tomography, biased data, statistical inverse problems, and network tomography. The conference was supported by the NSF Grant DMS 05-34181, and by the Faculty of Arts and Sciences and the Department of Statistics of Rutgers Univer- sity. We would like to thank the participants of the conference, the contributors to the volume, and the anonymous reviewers. Thanks are also due to DIMACS for providing conference facilities, and to the members of the staff and the many grad- uate students from the Department of Statistics for their tireless efforts to ensure the success of the conference. Last but not least, we would like to thank Ms. Pat Wolf for her patience and meticulous attention to all details in handling the papers in this volume. Regina Liu, William Strawderman and Cun-Hui Zhang December 15, 2006 v
- 10. vi Dedication This volume is dedicated to our dear colleague Yehuda Vardi, who passed away in January 2005. Yehuda was born in 1946 in Haifa, Israel. He earned a B.S. in Mathematics from Hebrew University, Jerusalem, an M.S. in Operations Research from the Technion, Israel Institute of Technology and a Ph.D. under Jack Kiefer at Cornell University in 1977. Yehuda served as a Scientist at ATT’s Bell Laboratories in Murray Hill before joining the Department of Statistics at Rutgers University in 1987. He served as the department chair from 1996 until he passed away. Yehuda was a dynamic and influential chair. He led the department with great energy and vision. He also provided much service to the statistical community by organizing many research conferences, and serving on the editorial boards of several statistical and engineering journals. He was an elected fellow of the Institute of Mathematical Statistics and International Statistical Institute. His research was supported by numerous grants from the National Science Foundation and other government agencies. Yehuda was a leading statistician and a true champion for interdisciplinary re- search. He developed key algorithms which are now widely used for emission tomo- graphic PET and SPECT scanners. In addition to his work on medical imaging, he coined the term “network tomography” in his pioneering paper on the problem of estimating source-destination traffic based on counts in individual links or “road sections” of a network. This problem has since blossomed into a full-fledged field of active research. His work on unbiased estimation based on biased data was a fundamental contribution in the field, and was recently rediscovered as a powerful general tool for the popular Markov chain Monte Carlo method. He has explored many other areas of statistics, including data depth and positive linear inverse problems with applications in signal recovery. His seminal contributions played a leading role in advancing the scientific fields in question, while enriching statistics with important applications. Yehuda was not just a scientist with remarkable breadth and insight. He was also a wonderful colleague and friend, and a constant source of encouragement and humor. We miss him deeply. Regina Liu, William Strawderman and Cun-Hui Zhang
- 11. Contributors to this volume Adhikari, A., Avaya Labs Bickel, P. J., University of California, Berkeley Cabrera, J., Rutgers University Cao, J., Bell Laboratories, Alcatel-Lucent Technologies Chang, I-S., National Health Research Institutes Chen, A., Bell Laboratories, Alcatel-Lucent Technologies Chien, L.-C., National Health Research Institutes Cohen, A., Rutgers University Denby, L., Avaya Labs Eyheramendy, S., Oxford University Fang, J., Rutgers University Gijbels, I., Katholieke Universiteit Leuven Hsiung, C. A., National Health Research Institutes Jornsten, R., Rutgers University Kolassa, J. E., Rutgers University Landwehr, J. M., Avaya Labs Lawrence, E., Los Alamos National Laboratory Li, B., Tsinghua University Lin, D., University of North Carolina López-Pintado, S., Universidad Pablo de Olavide Madigan, D., Rutgers University Mallows, C., Avaya Labs Mandel, M., The Hebrew University of Jerusalem Meloche, J., Avaya Labs Meyer, M., University of Georgia Michailidis, G., University of Michigan Nair, V. N., University of Michigan Naus, J., Rutgers University Pal, J. K., University of Michigan Rinott, Y., Hebrew University Sackrowitz, H., Rutgers University Shepp, L., Rutgers University Shlomo, N., Southampton University vii
- 12. viii Contributors to this volume Singh, K., Rutgers University Strawderman, W. E., Rutgers University Tang, W., Rutgers University Vardi, Y., Rutgers University Vitale, R. A., University of Connecticut Wen, C.-C., Tamkang University Woodroofe, M., University of Michigan Wu, Y.-J., Chung Yuan Christian University Xie, M., Rutgers University Ying, Z., Columbia University Yu, C.-R., Rutgers University Zhang, C.-H., Rutgers University Zhang, J., Rutgers University
- 13. IMS Lecture Notes–Monograph Series Complex Datasets and Inverse Problems: Tomography, Networks and Beyond Vol. 54 (2007) 1–11 c Institute of Mathematical Statistics, 2007 DOI: 10.1214/074921707000000021 Deconvolution by simulation Colin Mallows1 Avaya Labs Abstract: Given samples (x1, . . . , xm) and (z1, . . . , zn) which we believe are independent realizations of random variables X and Z respectively, where we further believe that Z = X + Y with Y independent of X, the problem is to estimate the distribution of Y . We present a new method for doing this, involving simulation. Experiments suggest that the method provides useful estimates. 1. Motivation The need for an algorithm arose in work on estimating delays in the Internet. We can send a packet from an origin A to a remote site B, and have a packet returned from B to A; the time that that this takes is called the “round-trip delay” for the link A-B. These delays are very volatile and are occasionally large. We can also send packets from A to a more remote site C, by way of B, and can arrange for packets to be returned from C via B to A; this gives the round-trip delay for the A-B-C path. However, we cannot directly observe the delay on the B-C link. Observation suggests that delays for successive packets are almost independent of one another; in particular the measured delays for two packets sent 20ms apart, the first from A to B (and return), the second from A to B to C (and return), are almost independent. We model this situation by assuming there are distributions FX and FY that give the delays on the links A-B and B-C respectively, with the distribution of the A-C delay being the convolution of these two distributions. In practice we are interested in identifying changes in the distributions as rapidly as possible. However a more basic question is, how to estimate the distribution FY when we can observe only X and Z? While our formulation of the deconvolution problem seems natural in our context, we have not seen any study of it in the literature. A Google Scholar search for titles containing “deconvolution” yields about 12000 references; many of these refer to “blind deconvolution” which is what a statistician would term “estimation of a transfer function”. If we delete titles containing “blind” there remain about 4730 titles. Most of these are in various applied journals, relating to a large variety of disciplines. A selection of those in statistical and related journals are listed in the References section. In all the papers we have seen, the distribution of X is assumed known. 2. A note on notation The usual convention is to write all mathematical variables in italics, with random variables in upper-case, and realizations in lower-case. We depart from this by using typewriter font like this for both observations and functions of them. Our 1Avaya Labs Basking Ridge, NJ, USA, e-mail: colinm@avaya.com AMS 2000 subject classifications: 60J10, 62G05, 94C99. Keywords and phrases: nonparametric estimation, Markov chains. 1
- 14. 2 C. Mallows algorithms are copied directly from implementations in the S language. Most of the S functions that we use are self-explanatory, but a detailed explanation appears in Appendix 1. Only two things need explanation here; the function c() (concatenate) makes its arguments into a vector. Also, many S functions take a vector argument. It is convenient that subscripts are not used in S; indices are shown by using square brackets. Thus a vector x of length 3 has elements x[1],x[2],x[3]. This notation makes it easy to write complicated expressions as indices. 3. Two naive methods, and a new idea Recall that the observed samples are x = c(x[1], . . . , x[m]) and z = c(z[1], . . . , z[n]). If we have m = n, a first suggestion is to sort x and z, forming sortx and sortz, and to form yhat ← sortz − sortx (i.e. yhat[i] = sortz[i] - sortx[i]). If the distributions of X and Z are Normal with variances σ2 and τ2 respectively, so that what we want is an estimate of a Normal distribution with variance τ2 − σ2 , this method produces an estimate of a Normal distribution with the correct mean but with variance (τ − σ)2 (because the sorted vectors are perfectly correlated), which is too small. The method is not consistent as n → ∞. Another approach, still assuming m=n, is to put both x and z into random orders and to compute the vector of differences z-x. Again, this does not work; this gives an estimate of the distribution of X + Y − X where X is an independent copy of X. In the Normal case described above, this method gives an estimate of a Normal distribution with variance τ2 + σ2 instead of τ2 − σ2 . The new idea is that a useful estimate could be obtained if we knew the “right” order in which to take the zs before subtracting the xs; and we can estimate an appropriate order by a simulation. Here is a first version of how this would work, assuming m=n. Suppose we have a first estimate of FY , represented by a vector of values oldy = c(oldy[1],...,oldy[n]). We choose a random permutation rperm of (1, . . . , n), and put the elements of oldy into this order. We add the xs to give a vector w where w ← x + oldy[rperm] We record the ranks of the elements of this vector. We put the elements of z into this same order and subtract the xs. Thus newy ← sort(z)[rank(w)] − x We can repeat this operation as many times as we like. We will attempt an explanation of why this might be expected to work below. An example is shown in Figure 1. Here the sample size is n = 100, and both X and Y are standard Normal. We generated pseudo-random samples z0 = x0 + y0 and x1, placed these in sorted order (sortz0 = sort(z0) and sortx1 = sort(x1) and started the algorithm by taking y1 = sort(sortz0 - sortx1). Successive versions of y were obtained using the iteration. Note that rank(runif(n)) is a random permutation of (1,. . . ,n). newy ← sort(sortz0[rank(sortx1 + oldy[rank(runif(n))])] − sortx1). We ran the iteration for 100 steps. Figure 1 shows the first nine y vectors, each sorted into increasing order, plotted against standard normal quantiles. Also shown is the straight line that corresponds to a normal distribution with mean mean(z0) - mean(x1) and variance var(z0) - var(x1). Figure 2 shows iterations 81:100.
- 15. Deconvolution 3 Fig 1. QQplots of the first nine iterations for the Normal example. Figure 3 shows values of a distance index d, which is the sum of absolute vertical deviations between this line and the estimate y. The algorithm appears to be stable, meaning that in repeated applications of the algorithm, the estimates stay close together. The initial transient takes no more than four iterations. The average value of the distance d over iterations 5:100 is 19.69. Also shown (with plotting character “o”) are comparable values of d for random samples from a normal distribution with the same mean and variance as this fitted normal distribution. The average value of these is 14.74. If we average the y vectors over iterations 5:100, we get a vector whose distance from this fitted normal distribution is d = 17.82. The average distance between the iterates and their average is d = 8.95. Thus the average distance between the iterates and their average is smaller than the average distance between random normal samples and the population line. The iteration seems to be giving good estimates of Y . Why should this be so? Here is an argument to support this expectation. Suppose z = x+y; these vectors are realizations of random variables X, Y, Z. we cannot observe any of x,y,z but can see sortz = sort(z) and an independent realization of X, namely x1. How can we define an estimate of y? Since X and Y are independent (by assumption), if rperm is a random permutation, then zhat = sort(x) + sort(y)[rperm] is a realization of Z, sorted according to sort(x). To retrieve y we simply subtract sort(x) from zhat. If n is large, we expect zhat to be close to z, and sort(x1) to be close to sort(x0). Thus we expect that putting z into the same order as zhat will make z approximately equal to zhat; and subtracting sort(x1) from this will approximately retrieve y. This argument does not explain why the iteration should converge when it is started with y0 remote from the correct value. We do not yet have an explanation of this.
- 16. 4 C. Mallows Fig 2. QQplots of iterations 81:100 for the Normal example. 4. Questions Several questions come to mind immediately. Is this algorithm always stable? Is the algorithm consistent, meaning that as n ← ∞, the empirical c.d.f of y converges in probability to FY ? I thank a referee for reminding me that FY may not be unique. What happens when it is not? To approach these questions, we point out that in the algorithm we have de- scribed, the possible values of the vector y are all of the form z[perm] − x where perm is a permutation of (1, . . . , n). Thus in repeated applications y executes a random walk on the n! possible values of this vector. This random walk will have a stationary distribution, which may not concentrate on a single state (this seems to be the usual case). Some states may be transient. Thus the most we can hope for is that this stationary distribution is close to FY in some sense. Clearly we need a proof that as n → ∞ this stationary distribution converges (in some sense) to a distribution that is FY whenever this is identifiable. Also it would be very pleasant to understand the distribution of the discrepancy measure d when y is drawn from the stationary distribution. As yet we do not have these results, but empirical evidence strongly suggests that the convergence result holds universally, and that useful estimates are obtained in all cases. However the dispersion among successive realizations of y is an over-optimistic estimate of the precision of the estimate of FY . Detailed analysis of the stationary distribution seems out of reach. Even with m = n = 3, 924 different configurations of x and z need to be considered. There are 208 distinct stationary distributions. See Appendix 2. We suggest that in practice we need to ignore an initial transient, and that the
- 17. Deconvolution 5 Fig 3. The index d for the first 100 iterations, with values for random normal samples. dispersion among successive realizations of y is an over-optimistic estimate of the precision of the estimate of FY . We need to consider how to handle boundary conditions, for example (as in the motivating example) that all values of Y are positive. The algorithm as stated need not generate vectors y that satisfy such conditions. Also, we question how the algorithm will perform when there are remote outliers in either or both z0 and x1. Since these samples are assumed to be independent of one another, there is no reason to hope that subtracting an x1 outlier from a z0 outlier will make any sense. We study these questions in Section 6 below. 5. Variations Several variations on the basic idea are as follows. (a) Instead of using the actual data (x[1],...,x[n]) use a sample from an estimate of FX, for example a bootstrap sample from the observed x. (b) To add some smoothness to the algorithm, at each iteration replace x by x + ξ and/or y by y + η, where ξ and η are vectors of small Gaussian perturbations. We can use the same perturbations throughout, or we can use independent perturbations at each step of the algorithm. (c) Similarly we can (independently) smooth z by adding ζ. If we arrange that varζ = varξ + varη, these smoothings should not introduce any bias into the estimate of FY , because X + ξ + Y + η is distributed like Z + ζ. Of course the efficiency of the method will degrade if the variance of ζ becomes large (unless each of X, Y, Z is Gaussian).
- 18. 6 C. Mallows If m and n are not equal, to apply the algorithm we need to generate equal numbers of values of x and z. We can do this either by (d) creating vectors of some length N by bootstrapping from the observed x and z (N could be very large, so that we are effectively regarding x and z as defining empirical distributions), (e) if m n, by taking z with a random sample (without replacement) from x; or similarly sampling z if n m; or (f) if mn, suppose n = km+r with rm. Then generate n values of x by repeating x k times and adjoining a random sample of size r drawn from x. Similarly if m n, repeat z to fill out m values. (g) In generating w we can use a bootstrap sample from y, possibly smoothed as above. To achieve stability in the estimate of FY , we can (h) Apply the algorithm a moderate number of times, k say, and average the resulting sorted y vectors; or (i) concatenate successive y vectors to form a pooled estimate of FY ; if we do this we can at each stage (j) generate w by sampling from this pooled estimate. It is not clear how to generalize the idea to deal with multivariate observations. 6. Boundary conditions, and outliers If some bound on Y is known a priori, for example if it is known (as in the motivating problem) that Y 0, we need to decide what to do if the algorithm produces one or more negative values in y. Some possibilities in this case are: (k) At each iteration, round negative values of yhat up to zero. (l) At each iteration, replace negative values by randomly sampling from the positive ones; (m) At each iteration, replace negative values in yhat by copies of the smallest values among the positive ones. (n) At each iteration, reject a random permutation if it leads to offending values; draw further permutations until one is obtained that satisfies the positivity conditions; (o) At each iteration, adjust the permutation by changing (at random) a few elements (as few as possible?) in such a way as to meet the conditions. Our experience so far suggests that none of these proposals works very well. Pro- posals (n) and (o) are excessively tedious, and have been tried only in very small examples. At this point we recommend another strategy, namely (p) Replace negative values in yhat by their absolute values. We investigated two of these proposals as follows. We generated 100 pseudo- random exponential variates x0, and added a similar (independent) vector y0 to form the observed vector z0. We assumed that an independent vector x1 was also observed. We ran the iteration in three ways: (q) no adjustment (l) replace negative values by a random sample from the positive values;
- 19. Deconvolution 7 Fig 4. The lowest 20 elements of the first nine iterations for each of three methods: Top:(q), Middle:(l), Bottom:(p). (p) Remove negative values of yhat, replacing them by their absolute values. This can be done in S by an application of the abs function: newy ← sort(abs(sortz0[rank(sortx1 +oldy[rank(runif(n))])] − sortx1)). All three methods performed similarly for values of yhat greater than 0.25. Figure 4 shows the lowest 20 values of yhat for the first nine iterations, plotted against standard exponential quantiles, for each of these three methods, together with the line through the origin with slope mean(z0) - mean(x1). We see that the naive method (q) produces a large number of negative values; method (l) avoids this but seems to introduce a positive bias; method (p) works well. Figure 5 shows the number of negative elements in yhat (before adjustment) for the three methods. The average numbers over the first 100 iterations are (q) 4.44, (l) 3.35, (p) 2.68. We have no understanding why the “absolute values” method works as well as it seems to. We have run similar trials for the case where both X and Y are uniform on (0, 1), so that Z has a triangular density supported on (0, 2). Here for method (p) we need to reflect values above y=1 to lie in (0, 1). Again, method (p) seems to be better than (q) and (l). We have also investigated the performance of the “absolute values” method when there is a positivity condition and outliers are present. We find that the non-outlying part of the distribution is estimated satisfactorily. We took x0, x1, and y0 each to contain 95 samples from a standard exponential distribution (with mean 1), and
- 20. 8 C. Mallows Fig 5. The number of negative elements (before adjustment) for each of three methods: Top:(q), Middle:(l), Bottom:(p). 5 samples from an exponential distribution with mean 100. Figure 6 shows the first four iterations of our basic algorithm, using the option (p) to adjust negative estimates, plotted against sort(y0). Figure 7 expands the lower corner of this plot, with the line through the origin of unit slope. The iterates seem to be staying close to this line. At this point our recommendation (if m = n and the variables are continuous, so that there are no ties in the computed values), is to use the original method, i.e. do not bootstrap or smooth or use (j). If the variables are lattice-valued, for example integer-valued, it seems to help to add small random perturbations to x and y[rperm] at each stage to break the ties randomly. It is not clear whether it is as good to simply add small perturbations once and for all. To handle the boundary and outlier problems, we recommend using the absolute-values method (p) above. Appendix 1. The S language In S the basic units of discourse are vectors; most functions take vector arguments. The elements of a vector x of length n are x[1], . . . , x[n]. c() is the “concatenate” function, which creates a vector from its arguments. Thus x = c(x[1], . . . , x[n]). If the elements of an m-vector s are drawn from 1, . . . , n, (possibly with repetitions), x[s] is the vector c(x[s[1]], . . . , x[s[m]]. The function sort() rearranges the ele- ments of its argument into increasing order; so if x = c(2,6,3,4), sort(x) is c(2,3,4,6). The function rank() returns the ranks of the elements of its argu- ment; i.e. rank(x)[i] is the rank of x[i] in x. Thus if x = c(2,6,3,4), rank(x)
- 21. Deconvolution 9 Fig 6. Four iterates of the basic algorithm, using option (p) to handle the positivity condition, when outliers are present. is c(1,4,2,3). sort(x)[rank(x)] is just x. Another function we have used is runif(), which generates pseudo-random uniform variables drawn from the in- terval (0,1). Thus rank(runif(n)) is a random permutation of 1, . . . , n rnorm(n) generates n standard normals; rexp(n) generates n random exponentials. The func- tion abs() replaces the elements of its argument by their absolute values. Appendix 2. The case m=n=3 Without loss of generality we may assume x1 = c(0,x,1) and z0 = c(-a,0,b) with 0 x 1/2 and a and b positive. Examination of the 36 possible values of (z0[perm1]-x1)[perm2] +x1 shows that the stationary distribution will change whenever any of a,b and a+b crosses any of the values x,2x,1,1+x,1-x,1-2x,2, 2-x,2-2x. For a general x in (0,1/2) these cut-lines divide the positive quadrant of the a,b plane into 154 regions. The configuration of these regions changes when x passes through the values (1/6,1/5,1/4,1/3,2/5). Thus we need to consider six representative values of x, perhaps x = c(10,22,27,35,44,54)/120, and for each of these values of x we have 154 regions, 924 regions in all. We computed the transition matrix of the random walk for each of these 924 cases, and found 208 different stationary distributions. One of these, where one state is absorbing and the other five transient, occurs 84 times. Ten distributions occur only once each. A similar calculation for m or n larger than 3 seems impractical. Acknowledgments. Thanks to Lorraine Denby, for showing me the problem, and to Lingsong Zhang, who did some of the early simulations. Also to Jim Landwehr,
- 22. 10 C. Mallows Fig 7. Expansion of the lower corner of Figure 6. Jon Bentley and Aiyou Chen for stimulating comments. Two referees contributed insightful remarks. References [1] Carroll, R. J. and Hall, P. (1988). Optimal rates of convergence for deconvolving a density. J. Amer. Statist. Assoc. 83 1184–1186. [2] Fan, J. (1991). On the optimal rates of convergence for nonparametric decon- volution problems. Ann. Statist. 19 1257–1272. [3] Fan, J. (2002). Wavelet deconvolution. IEEE Trans. Inform. Theory 48 734– 747. [4] Goldenshluger, A. (1999). On pointwise adaptive nonparametric deconvo- lution. Bernoulli 5 907–925. [5] Jansson, P. A., ed. (1997). Deconvolution of Images and Spectra. Academic Press, New York. [6] Liu, M. C. and Taylor, R. L. (1989). A consistent nonparametric density estimator for the deconvolution problem. Canad. J. Statist. 17 427–438. [7] Mendelsohn, J. and Rice, J. (1982). Deconvolution of microfluorometric histograms with B-splines. J. Amer. Statist. Assoc. 77 748–753. [8] Starck, J.-L. and Bijaoui, A. (1994). Filtering and deconvolution by the wavelet transform. IEEE Trans. Signal Processing 35 195–211. [9] Stefanski, L. A. and Carroll, R. J. (1990). Deconvoluting kernel density estimators. Statistics 21 169–184. [10] Zhang, C. H. (1990). Fourier methods for estimating mixing densities and distributions. Ann. Statist. 18 806–830.
- 23. Deconvolution 11 [11] Zhang, H. S., Liu, X. J. and Chai, T. Y. (1997). A new method for optical deconvolution. IEEE Trans. Signal Processing 45 2596–2599.
- 24. IMS Lecture Notes–Monograph Series Complex Datasets and Inverse Problems: Tomography, Networks and Beyond Vol. 54 (2007) 12–23 c Institute of Mathematical Statistics, 2007 DOI: 10.1214/074921707000000030 An iterative tomogravity algorithm for the estimation of network traffic Jiangang Fang1 , Yehuda Vardi1,∗ and Cun-Hui Zhang1,† Rutgers University Abstract: This paper introduces an iterative tomogravity algorithm for the estimation of a network traffic matrix based on one snapshot observation of the link loads in the network. The proposed method does not require complete observation of the total load on individual edge links or proper tuning of a penalty parameter as existing methods do. Numerical results are presented to demonstrate that the iterative tomogravity method controls the estimation error well when the link data is fully observed and produces robust results with moderate amount of missing link data. 1. Introduction This paper concerns the estimation of network traffic based on link data. The traf- fic matrix of a network, which gives the amount of source-to-destination (SD) flow, is an essential element in a wide range of network administration and engineering applications. However, in today’s fast growing communication networks, it is of- ten impractical to directly measure network traffic matrices due to cost, network protocol and/or administrative constraints, while measurements of the total traffic passing through certain individual links are more readily available. Thus, the prob- lem of estimating SD traffic based on link data, called network tomography [10], is of great interest to communications service providers. In the network tomographic model [10] y = Ax, (1.1) where y is a vector of traffic loads on links, A = (aij) is a known routing matrix with elements aij = 1 if link i is in the path for the j-th pair of SD nodes and aij = 0 otherwise, and x is the SD traffic flow as a vectorization of the traffic matrix. Here the routing protocol A is fixed. In typical network applications, the number of links (edges) is of the same order as the number of nodes (vertices) in the network graph, while the number of SD pairs is of the order of the square of the number of nodes. Thus, dim(y) dim(x) and the network tomographic model (1.1) is ill-posed. Vardi [10] identified the ill-posedness of (1.1) as the main difficulty of network tomography and proposed to estimate the expected traffic flow based on ∗Research partially supported by National Science Foundation Grant DMS-0405202. †Research partially supported by National Science Foundation Grants DMS-0405202, DMS- 0504387 and DMS-0604571. 1Department of Statistics, Hill Center, Busch Campus, Rutgers University, Piscat- away, New Jersey 08854, USA, e-mail: alexfang@stat.rutgers.edu; vardi@stat.rutgers.edu; czhang@stat.rutgers.edu AMS 2000 subject classifications: 62P30, 62H12, 62G05, 62F10. Keywords and phrases: network traffic flow, network tomography, Kullback-Leiber distance, network gravity model, regularized estimation. 12
- 25. Iterative tomogravity algorithm 13 independent copies of y by modeling the variance of y. The problem has since being considered by many research groups. See Vanderbei and Iannone [9] for MLE/EM, Cao et al. [1, 2] for MLE/EM in the model xj ∼ N(λj, φλc j) and non-stationarity issues, Medina et al. [8], Liang and Yu [6] for a more scalable pseudo-likelihood, Liang et al. [7] for additional direct observations of flow data for selected SD pairs, and Coates et al. [4] and Castro et al. [3] for surveys with additional references. In general, these methods require observations of multiple copies of y. An interesting and noticeable development in the area is the introduction of (tomo)gravity algorithms and related methods based on a single snapshot of the network, i.e. one copy of y. Zhang et al. [11] observed that in certain communica- tions networks (e.g. a backbone network where each node represents a PoP, or point of presence), almost all the traffic flow is generated by and destined to a known set of edge nodes which do not serve as intermediate nodes in any SD paths. Thus, each SD path begins with a source edge node, traverses through an inbound edge link, an inner network, and then an outbound edge link to a destination edge node. Under this assumption, the total inbound flow N (in) s from a source node s is the sum of the loads over all the inbound edge links from s and the total outbound flow N (out) d to a destination node d is the sum of the loads over all the outbound edge links to d. The edge nodes communicate to each other through an inner network with a directed graph composed of inner nodes and links and a routing protocol, but the inner nodes does not generate or receive traffic. Moreover, Zhang et al. [11] observed that for each fixed source node s, the distribution of the inbound traffic N (in) s from s to different destinations d is approximately proportional to the total outbound loads N (out) d these destinations receive. Formally, this is called the gravity model and can be written in Vardi’s [10] vectorization as xj = N (in) sj N (out) dj N , N = s N(in) s = d N (out) d , (1.2) where sj and dj are respectively the source and destination nodes for the j-th SD pair, xj is the corresponding component of the simple gravity solution x as an approximation of the vector x in (1.1), and N is the total flow. The gravity model is best described as xsd = N(in) s N (out) d /N (1.3) with a slight abuse of notation, where xsd is the traffic flow from source s to des- tination d in the gravity model, i.e. xj = xsj dj . Here, the relationship between the link data y and the SD traffic flow x is still governed by the tomographic model (1.1). Due to the additional information provided in the gravity model (1.2) about the nature of the SD traffic x, the number of unknowns in x is square rooted. Thus, the ill-posedness of (1.1) is greatly alleviated. In particular, if all link loads are ob- served, the total inbound flow N (in) s and outbound flow N (out) d for individual edge nodes and thus the total traffic N are all available network statistics in the gravity model. In addition to the gravity solution x in (1.2), Zhang et al. [11] developed the simple tomogravity solution arg min x x − x : Ax = y (1.4) and more general tomogravity solutions when the edge nodes are further classified as “access” or “peering”, while Zhang et al. [12] developed entropy regularized
- 26. 14 J Fang, Y. Vardi and C.-H. Zhang tomogravity solution as arg min x y − Ax2 + φN2 K x/N, x/N , (1.5) where K(·, ·) is the Kullback-Leibler information and φ is a tuning parameter for the penalty level. These tomogravity solutions require complete knowledge of the total inbound and outbound flow, i.e. N (in) s and N (out) d , for all individual source and destination nodes. They perform reasonably well when such information is available and have been implemented in certain ATT commercial networks. In this paper, we propose an iterative tomogravity (ITG) algorithm which alter- nately seeks estimates as local optimal solutions in the tomographic space (1.1) and a gravity space of network traffic flow x. Our algorithm, described in Section 2, is based on a single snapshot of the link data and does not require the full knowledge of the total inbound and outbound flow for all individual edge nodes. The idea is to use the gravity space, instead of the specific simple gravity solution (1.2), to reg- ularize the network tomography problem (1.1). In Section 3, we present the results of a real-data experiment to demonstrate that the ITG method is competitive com- pared with other tomogravity algorithms when the complete link data is available and robust when a moderate amount of link data is missing. 2. An iterative tomogravity algorithm In a general network tomographic model, the observed link data, as a sub-vector y∗ of the vector y in (1.1), satisfies y∗ = A∗ x, (2.1) where the matrix A∗ = (a∗ ij) is composed of the rows of the routing matrix A in (1.1) corresponding to the observed links, and x is the SD traffic flow as in (1.1). Let J be the total number of SD-pairs of concern. For the observation y∗ , the tomographic space of probability vectors is T ∗ = f ∈ I RJ : y∗ ∝ A∗ f, f ≥ 0, 1T f = 1 , (2.2) where 1 is the vector composed of 1’s and vT denotes the transpose of a vector v. Here and in the sequel, inequalities are applied to all components of vectors. In the literature, different types of flow and load are often specifically denoted. Let y(net) be the link loads of the inner network, y(edge) the loads on the links be- tween the edge nodes and inner network, y(self) the load on the links from the edge nodes to themselves, x(net) the traffic flow between distinct edge nodes (necessarily through the inner network), and x(self) the flow of the edge nodes to themselves. Since x(self) does not go through the inner network and the flow from an edge node to itself is the same as the load on the corresponding self-link, the tomographic model can be written as y = y(net) y(edge) y(self) = A(net) 0 A(edge) 0 0 I(self) x(net) x(self) = Ax, (2.3) with I(self) being the identity matrix giving y(self) = x(self) , provided that the inner network does not generate traffic. This is a special case of Vardi’s [10] tomographic
- 27. Iterative tomogravity algorithm 15 model (1.1) describing decompositions of the SD traffic x and link load y, but (1.1) can be also viewed as y(net) = A(net) x(net) . In this paper, the observed y∗ in (2.1) is a general sub-vector of the y in (2.3) to allow partial observation of y(edge) and networks without y(self) and x(self) . Suppose throughout the sequel that the list of the SD-pairs (sj, dj), i = 1, . . . , J, forms a product set composed of all the pairings from a set S of source nodes to a set D of destination nodes (D = S allowed), so that J = |S||D|, where |C| is the size of a set C. This gives a one-to-one mapping between I RJ and the space of all |S| × |D| matrices: v = (v1, . . . , vJ )T ∼ (vsd)|S|×|D|, vj = vsj dj . In this notation, the gravity space of probability vectors is G = g ∈ I RJ : g ∼ (gsd)|S|×|D| = p qT , g ≥ 0, 1T g = 1 , (2.4) i.e. gsd = psqd or matrices of rank 1, where p ∈ I R|S| and q ∈ I R|D| . Zhang et al. [11] proposed (1.2) as the simple gravity algorithm and (1.4) as the simple tomogravity algorithm. Zhang et al. [12] proposed (1.5) as the entropy- regularized tomogravity algorithm. Their basic ideas can be summarized as follows: (i) The gravity model gives a rough approximation of the SD flow; (ii) When the simple gravity solution (1.2) is available, it can be used to regularize Vardi’s tomo- graphic model (1.1). Motivated by their work, we propose the following algorithm which provides estimates of the SD flow x in (2.1). Iterative tomogravity algorithm (ITG): Initialization: g = 1/J (2.5) Iteration: f(new) = arg min K(f, g(old) ) : f ∈ T ∗ (2.6) g(new) = arg min K(f(new) , g) : g ∈ G (2.7) Finalization: N = 1T y∗ 1T A∗ f( fin) (2.8) x = Nf( fin) (2.9) where K(f, g) is the Kullback-Leibler information defined as (2.10) K(f, g) = J j=1 fj log fj gj . As mentioned earlier, our basic idea is to use the gravity space (2.4), instead of the simple gravity solution (1.2), to regularize the tomographic model (2.2). A main advantage of this approach is that it does not require the knowledge of the simple gravity solution or equivalently, the complete observation of loads on all edge links. Numerical results in Section 3 demonstrate that when the complete link data y is observed, the ITG (2.9) and the entropy-regularized tomogravity (1.5) perform comparably in terms of estimation error, and they both outperform the simple gravity (1.2) and tomogravity (1.4). Moreover, the ITG without using the knowledge of the “access” or “peering” status of links has similar performance compared with the generalized tomogravity method [11] which requires such knowledge. We note that the ITG method does not need a tuning parameter as (1.5) does.
- 28. 16 J Fang, Y. Vardi and C.-H. Zhang A main difference between ITG (2.9) and the simple tomogravity (1.4) is that the simple gravity solution x in (1.2) is not explicitly used in ITG, since g is treated as an unknown in the ITG algorithm. However, the information in the observed portions of y(edge) and y(self) is still utilized in the ITG iterations through the tomographic space (2.2), instead of directly computing x from y(edge) and y(self) as in (1.3). If the simple gravity x (or an approximation of it if x is not fully available) is used as the initialization for ITG, the simple tomogravity solution is the result of a single ITG iteration. We may also treat g = x/N as an unknown in (1.5), cf. Section 4, but then a tuning parameter is still required. We use the relaxation algorithm of Krupp (1979) to compute (2.6) of the ITG, while (2.7) is explicit with g (new) sd = d f (new) sd s f (new) sd as in (1.3). Here is a full description of the relaxation algorithm. Let g(old) = (g (old) 1 , . . . , g (old) J )T be a given probability vector. The problem is to minimize K f, g(old) under the linear constraints in (2.2). Since y∗ i = 0 implies fij = 0 for all j with a∗ ij = 1 and thus reduces the optimization problem to a subset of j, we assume y∗ = (y∗ 1, . . . , y∗ r ) 0 where r is the total number of links with observed load. Define hij = a∗ ij/y∗ i − a∗ rj/y∗ r , i = 1, . . . , r − 1, 1, i = r. The linear constraints A∗ f = y∗ and 1T f = 1 for the tomographic space (2.2) can be written as Hf = (0T , 1)T , where H = (hij). Krupp’s [5] relaxation algorithm maximizes vr − J j=1 g (old) j exp r i=1 hijvi − 1 (2.11) over all vectors v = (v1, . . . , vr)T and then set f (new) j = g (old) j exp r i=1 hijvi − 1 . (2.12) As (2.11) is concave in v, its optimization is done by the Newton-Raphson method for individual components vi, cycling through i = 1, . . . , r. Since hr,j = 1 for all j, f(new) in (2.12) is properly normalized. The iteration steps (2.6) and (2.7) are both monotone in K(f, g), so that the ITG algorithm reaches a local minimum of the Kullback-Leibler information between the tomographic (2.2) and gravity (2.4) spaces. However, since K(f, g) is not convex jointly in (f, g) with g in the gravity space, ITG is not guaranteed to converge to a global minimum. 3. An example We conduct numerical experiments with data collected over the Abilene Network (an Internet2 high-performance backbone network in United States) illustrated in
- 29. Iterative tomogravity algorithm 17 Fig 1. Abline Network. Figure 1, with 12 nodes, 144 total traffic pairs (132 SD pairs and 12 self pairs), 30 inner links, and 24 edge links. We collect the full 12 × 12 SD traffic matrices in 5 min intervals for consecutive 19 weeks in 2004. We randomly pick four different periods of 3 days and use the data in these four time periods. We call these four raw datasets as X1, X2, X3, and X4. It turns out that the four datasets give different traffic patterns as the time periods cover different days of the week, cf. Figure 2. For each dataset and each hour, we compute x as the hourly total SD flow and y = Ax with a fixed routing matrix A used in the Abilene data. We compare four procedures using the complete data y as y∗ : the ITG (2.9), the simple tomogravity (STG) in (1.4), the generalized tomogravity (GTG) of Zhang et al. [11] utilizing the extra information of “access” or “peering” status of links, and the entropy regularized tomogravity (ERTG) in (1.5). Since the traffic flow for self pairs (s = d) is directly observable as the load on the self links, the ITG and STG estimate these components of x without error. Thus, we measure the performance of all estimators by the relative total error for non-self SD pairs s=d xsd − xsd s=d xsd, (3.1) where xsd is the flow from source s to destination d. We compute the relative total error for (1.5) with various values of the tuning parameter φ and found that the Table 1 Average of relative total errors for 288 different hours (4 different 3-day periods) based on complete link data. The best tuning parameter is used for the ERTG, while extra information is used for the GTG. Method risk Iterative Tomogravity (ITG) 0.3001 Entropy Regularized (ERTG) 0.2995 Simple Tomogravity (STG) 0.3139 Generalized Tomogravity (GTG) 0.3026
- 30. 18 J Fang, Y. Vardi and C.-H. Zhang Fig 2. The total hourly traffic for the 4 non-overlapping 3 day periods. Fig 3. Compare of the error rate using different models, dataset X1.
- 31. Iterative tomogravity algorithm 19 Fig 4. Compare of the error rate using different models, dataset X2. Fig 5. Compare of the error rate using different models, dataset X3.
- 32. 20 J Fang, Y. Vardi and C.-H. Zhang Fig 6. Compare of the error rate using different models, dataset X4. performance of (1.5) is near the best in a wide neighborhood of φ = 10−3 = 0.001. This confirms the results of Zhang et al. [12]. Thus, φ = 10−3 = 0.001 is used for (1.5) in our experiment. We plot the relative total error (3.1) against hour for the four datasets in Figures 3, 4, 5, and 6. We tabulate the average relative error in Table 1. From the results of the experiments, we observed that the performance of the proposed ITG is comparable to the ERTG with the best choice of the tuning parameter and the GTG based on extra information, while all three outperform the STG. We also exam the relative errors for different SD pairs as functions of the total traffic flow for the SD pairs. We compute the relative total error over 3-day time periods t∗ t=1 x (t) sd − x (t) sd t∗ t=1 x (t) sd (3.2) for fixed SD pairs in individual datasets, where t indicates time points with t∗ = 72. We group the values of (3.2) for SD pairs in all datasets according to the total flow t∗ t=1 x (t) sd with the grid {0, 1/4, 1/2, 3/4, 1, 1.5, 2, 2.5, 3, 4, 5, 7} in the unit of 1010 packets, and tabulate in Table 2 the average of (3.2) within groups. From Table 2, we observe that the estimation error is essentially a decreasing function of the amount of traffic for individual SD pairs. Finally, we check the robustness of the ITG (2.9) with missing link data (i.e. y∗ is a proper sub-vector of y). We focus on the case of missing data in edge links as the ITG is the only procedure among the four that do not require observations for all edge links. Let k be the number of edge links with missing data. We use only
- 33. Iterative tomogravity algorithm 21 Table 2 Relative total errors over 72 hours for fixed SD pairs and 3-day periods, grouped according to the total flow. The relative total errors are decreasing functions of the flow for all 4 procedures. Flow # in ITG ERTG STG GTG Level Group 0 – 0.25 215 4.4799 5.8725 4.5833 5.3545 0.25 – 0.5 100 0.4320 0.4279 0.4548 0.4158 0.5 – 0.75 73 0.3457 0.3449 0.3666 0.3467 0.75 – 1 30 0.2997 0.2992 0.3379 0.2505 1 – 1.5 46 0.2286 0.2305 0.2402 0.2588 1.5 – 2 25 0.2878 0.2859 0.2934 0.3089 2 – 2.5 18 0.1836 0.1828 0.1802 0.2080 2.5 – 3 6 0.1583 0.1576 0.1689 0.1207 3 – 4 7 0.1143 0.1126 0.1335 0.1261 4 – 5 6 0.1456 0.1448 0.1514 0.1373 5 – 7 2 0.0887 0.0938 0.0767 0.0882 data for the first day in dataset X1 and compute the average of the relative total error for 10 random missing patterns for each given k. We plot this average against k in Figure 7. From Figure 7, we find that the performance of the ITG method is robust against small or moderate amount of missing link data (up to 5 out of 24 edge links). 4. Discussion We consider the estimation of SD traffic flow in a network based on observations of a snapshot of traffic loads on links. Based on the ideas of Vardi [10] and Zhang et al. [11, 12], we propose an iterative tomogravity method which allows incomplete observation of the link data. Our main idea is to use the gravity space (2.4), instead of the simple gravity solution (1.2), to regularize Vardi’s [10] tomographic model (1.1). A numerical study with a real-life dataset demonstrates that the proposed method has similar performance compared with the methods proposed in [11, 12] which demand complete observation of the link data. We discuss below a number of related issues. There are two other possible ways of using the gravity space (2.4) to regularize (1.1) that we do not explore in this paper. The first is to use the ITG (2.9) instead of the simple gravity (1.2) in the penalty function in (1.5), resulting in arg min x y∗ − A∗ x2 + φ N2 K x/ N, g(fin) / N (4.1) with the N in (2.8). The second is to alternate between the optimization in the gravity space and entropy-regularized solution, i.e. to replace (2.6) with N(new) = 1T y∗ 1T A∗ g(old) (4.2) f(new) = arg min y∗ − N(new) A∗ f2 +φ{N(new) }2 K f, g(old) : f ∈ T ∗ . (4.3) A small numerical study seems to indicate that there is little difference between (4.1) and the ITG. The proposed ITG (2.9) implicitly assumes that the measurement error in the tomographic model (2.1) is of smaller order than the bias representing the Kullback- Leibler distance K(x/N, G∗ ) between x/N and the gravity space (2.4). This seems
- 34. 22 J Fang, Y. Vardi and C.-H. Zhang Fig 7. Relative total errors of the ITG versus the number of edge links with missing data. Average over 10 random missing patterns is used for each point in the plot. The ITG is robust against small or moderate amount of missing link data. to be the case in our real-data experiments since ITG significantly improves upon the simple tomogravity (1.4) by formally reducing K(x/N, x/N) to K(x/N, G∗ ). In cases where the measurement error in the tomographic model is potentially of larger order than K(x/N, G∗ ) [or K(x/N, x/N)] it would make sense to replace (2.6) by (4.2) and (4.3) in ITG [or to use (1.5)] with a proper tuning parameter φ. A possibility to further reduce the bias is to consider the mixed gravity model Fmix = f : f = k∗ k=1 πkf(k) , f(k) ∈ G . (4.4) For example, we may compute a regularized mixed tomogravity solution arg min y − N k∗ k=1 πkf(k) + N2 k∗ k=1 φkK(f(k) , g(k) ) (4.5) by alternately optimizing over g(k) ∈ G, f(k) , k = 1, . . . , k∗ and the mixing vector (π1, . . . , πk∗ )T . It seems that for a network with a fixed routing protocol, the ITG estimate x in (2.9) is a continuous map of y∗ , so that x − Ex is asymptotically normal when y∗ − EA∗ x is asymptotically normal with Ex/N ∈ G, as N → ∞. Our simulation study in a small artificial network has demonstrated the validity of this asymptotic normality theorem for moderate sample sizes. Estimation of traffic matrix based on link-load data alone is difficult as the estimation error is typically above 20%. More accurate results can be obtained if
- 35. Iterative tomogravity algorithm 23 additional information can be extracted from packets passing through routers. See for example Zhao, Kumar, Wang and Xu [13]. References [1] Cao, J., Davis, D., Vander Weil, S. and Yu, B. (2000). Time-varying network tomography: Router link data J. Amer. Statist. Assoc. 95 1063–1075. [2] Cao, J., Vander Wiel, S., Yu, B. and Zhu, Z. (2000). A scalable method for estimating network traffic matrices. Technical report, Bell Labs. [3] Castro, R. Coates, M., Liang, G., Nowak, R. and Yu, B. (2004). Net- work tomography: Recent developments. Statist. Sci. 19 499–517. [4] Coates, M. and Nowak, R. (2002). Sequential Monte Carlo inference of internal delays in nonstationary communication networks. IEEE Trans. Signal Process. 50 366–376. [5] Krupp, R. S. (1979). Properties of Kruithof’s projection method. The Bell System Technical J. 58 517–538. [6] Liang, G. and Yu, B. (2003). Maximum pseudo-likelihood estimation in net- work tomography. IEEE Trans. Signal Process. 51 243–253. [7] Liang, G., Taft, N. and Yu, B. (2006). A fast lightweight approach to origin- destination IP traffic estimation using partial measurements. Special Issue of IEEE-IT and ACM Networks on Data Networks, January 2006. [8] Medina, A., Taft, N., Salamatian, K., Bhattacharyya, S. and Diot, C. (2002). Traffic matrix estimation: Existing techniques compared and new directions. SIGCOMM, Pittsburgh, Aug. 2002. [9] Vanderbai, R. J. and Iannone, J. (1994). An EM approach to OD matrix estimation. Technical Report SOR 94-04, Princeton Univ. [10] Vardi, Y. (1996). Network tomography: Estimating source-destination traffic intensities from link data. J. Amer. Statist. Assoc. 91 365–377. [11] Zhang, Y., Roughan, M., Duffield, N. and Greenberg, A. (2003). Fast accurate computation of large-scale IP traffic matrices from link loads. In ACM SIGMETRICS, San Diego, USA, June 2003. [12] Zhang, Y., Roughan, M., Lund, C. and Donoho, D. (2003). An information-theoretic approach to traffic matrix estimation. In ACM SIG- COMM, Karlsruhe, Germany, August 2003. [13] Zhao, Q., Kumar, A., Wang, J. and Xu, J. (2005). Data streaming algo- rithms for accurate and efficient measurement of traffic and flow matrices. ACM SIGMETRICS, Banff, Canada, June 2005.
- 36. IMS Lecture Notes–Monograph Series Complex Datasets and Inverse Problems: Tomography, Networks and Beyond Vol. 54 (2007) 24–44 In the Public Domain DOI: 10.1214/074921707000000049 Statistical inverse problems in active network tomography Earl Lawrence1,∗ , George Michailidis2,∗ and Vijayan N. Nair2,∗ Los Alamos National Laboratory and University of Michigan Abstract: The analysis of computer and communication networks gives rise to some interesting inverse problems. This paper is concerned with active network tomography where the goal is to recover information about quality-of-service (QoS) parameters at the link level from aggregate data measured on end-to- end network paths. The estimation and monitoring of QoS parameters, such as loss rates and delays, are of considerable interest to network engineers and Internet service providers. The paper provides a review of the inverse problems and recent research on inference for loss rates and delay distributions. Some new results on parametric inference for delay distributions are also developed. In addition, a real application on Internet telephony is discussed. 1. The inverse problems Consider a topology with a tree structure defined as follows: T = {V, E} has a set of nodes V and a set of links or edges E. Figure 1 shows two examples, a simple two-layer symmetric binary tree on the left and a more general four-layer tree on the right. Each member of E is a directed link numbered after the node at its terminus. V includes a (single) root node 0, a set of receiver or destination nodes R, and a set of internal nodes I. The internal nodes have a single incoming link and at least two outgoing links (children). The receiver nodes have a single incoming link but no children. For the tree on the right panel of Figure 1, R = {2, 3, 6, 8, 9, 10, 11, 12, 13, 14, 15} and I = {1, 4, 5, 7}. All transmissions are sent from the root (or source) node to one or more of the receiver nodes. This generates independent observations Xk at all links along the paths to those receiver nodes. Let X denote this set of measurements. These data are not directly observable; rather we can collect only end-to-end data at the receiver nodes: Yr = f(X) for r ∈ R. The statistical inverse problem is to reconstruct the distributions of the link-level Xks from these path-level measurements. Examples of f(·) are: f(X) = k∈P(0,r) Xk, f(X) = k∈P(0,r) Xk, and f(X) = mink∈P(0,r) Xk, and f(X) = maxk∈P(0,r) Xk, where P(0, r) is the path between the root node 0 and the receiver node r. In this paper, we will be concerned only with the first two cases of f(·) above. To understand the statistical issues and challenges involved, let us examine some simple examples. ∗The research was supported in part by NSF Grants CCR-0325571, DMS-0204247 and DMS- 0505535. 1Statistical Sciences Group, Los Alamos National Laboratory, Los Alamos NM 87545, USA, e-mail: earl@lanl.gov 2Department of Statistics, University of Michigan, Ann Arbor MI 48109, USA, e-mail: gmichail@umich.edu; vnn@umich.edu AMS 2000 subject classifications: 62F10, 60G05, 62P30. Keywords and phrases: Network tomography, internet, inverse problems, monitoring, nonlinear least squares. 24
- 37. Inverse problems in network tomography 25 Fig 1. Examples of tree network topologies. A binary two-layer tree is shown on the left panel and a general four-layer tree on the right panel. The path lengths from the root to nodes belonging to the same layer are the same. Example 1. Consider the two-layer binary tree on the left panel of Figure 1, and suppose the Xk are binary with P(Xk = 1) = αk, k = 1, 2, 3 for the three links. Further, the root node sends transmissions to the receiver nodes one at a time. Take f(a, b) = ab. Then, the observed data are Y2j = X1jX2j for transmission j and Y3m = X1mX3m for transmission m. They are independent Bernoulli with prob- abilities α1α2 and α1α3, respectively. Suppose we send M transmissions to receiver node 2 and N transmissions to receiver node 3. Let M1 and N1 be the respective number of “ones”. Then, M1 and N1 are independent binomial random variables with success probabilities α1α2 and α1α3. From these data, we can estimate only α1α2 and α1α3. The individual link-level parameters α1, α2 and α3 cannot be fully recovered. Example 2. Take the same two-layer binary tree with binary outcomes with f(a, b) = ab as above. But now the root node sends transmissions to receiver nodes 2 and 3 simultaneously. In other words, the m-th transmission generates random variables X1m, X2m and X3m on all of the links. We observe Y2m = X1mX2m and Y3m = X1mX3m. The distinction from Example 1 is that the X1m is com- mon to both Y2m and Y3m. Now, each transmission has 4 possible outcomes: (1, 1), (1, 0), (0, 1), (0, 0) depending on whether the transmission reaches none, one, or both of the receiver nodes. If we send N such transmissions to nodes 2 and 3 simultaneously, the result is a multinomial experiment with probabilities α1α2, α1(1 − α2), (1 − α1)α2, and (1 − α1)(1 − α2) corresponding to the four outcomes. Let N(i, j) denote the number of events with outcome (i, j). Then, E[N(1, 1)] = α1α2α3, E[N(1, 1) + N(1, 0)] = α1α2, and E[N(1, 1) + N(0, 1)] = α1α3. It is easy to see that we can estimate all the three link-level parameters from these measure- ments. Thus, the data transmission scheme plays an important role in this type of
- 38. 26 E. Lawrence, G. Michailidis and V. N. Nair inverse problems. Example 3. Again we have a two-layer binary tree but now f(a, b) = a + b. Then Y2 = X1 + X2 and Y3 = X1 + X3. Let Fk be the distribution of the link-level random variables Xk ∈ R, for k = 1, 2, 3. Assume, as in Example 2, that the root node sends transmissions simultaneously to both receivers. In this case, even with simultaneous transmission to both receivers, the link-level parameters are not always identifiable. Just take Xk to be independent Normal(µk, 1), k = 1, 2, 3. Then Y2 and Y3 are bivariate normal with mean µ1 + µ2 and µ1 + µ3, variance 2 and correlation 1. One can see that the individual µk cannot be recovered from the joint distribution of Y2 and Y3. Additional assumptions on the distribution are needed in order to solve the inverse problem. We will revisit this issue. Example 4. Consider now the more general tree on the right panel of Figure 1. Again, we send transmissions to all of the receiver nodes simultaneously. If the ran- dom variables are binary and f(x1, . . . , xp) = p j=1 xj, all the link-level parameters are identifiable. The same is true for a general Xk with f(x1, . . . , xp) = p j=1 xj under suitable conditions on the distribution of the Xk (as discussed later in the paper). However, it may be “expensive” to send transmissions to all receiver nodes simultaneously. Instead, can we schedule transmissions to some judicious subsets of the receiver nodes at a time and combine the information appropriately to estimate all the link-level parameters? It is clear from Example 1 that it is not enough to send transmissions to one receiver node at a time. How should the transmission scheme be designed in order to estimate all the parameters? Are there some “good” schemes (according to some appropriate criteria)? These examples are simple instances of issues that arise in the context of ana- lyzing computer and communications networks and are collectively referred to as active network tomography. In the next section, we will describe the network ap- plication and the need for estimating quality-of-service (QoS) parameters such as loss rates and delays. Section 3 provides an overview of recent results in the lit- erature on the design of transmission experiments and inference for loss rates and discrete delay distributions. A real application on data collected from the campus network at the University of North Carolina, Chapel Hill is used to illustrate some of the results. Section 4 develops some new results on parametric inference for delay distributions. 2. Active network tomography The area of network tomography originated with the pioneering work of Vardi [14] where the term was first introduced. His work dealt with another type of in- verse problem relating to origin-destination (OD) traffic matrix estimation. The OD information is important in network management, capacity planning, and pro- visioning. In this problem, one is interested in estimating the intensities of traffic flowing between the origin-destination pairs in the network. However, we cannot collect these data directly; rather, one places equipment at the individual nodes (routers/switches) and collects aggregate data on all traffic flowing through the nodes i ∈ V. The goal is to recover distributions of origin-destination traffic be- tween all pairs of nodes in the network. There has been considerable work in this area, and a summary of the developments can be found in [3]. Active network tomography, on the other hand, is concerned with the “opposite” problem of estimating link-level information from end-to-end data. One sends test
- 39. Inverse problems in network tomography 27 probes (packets) (active probing) from a source to one or more receiver nodes on the periphery of the network and gets end-to-end path-level data on losses and delays. One then has to solve the inverse problem of reconstructing link-level loss and delay information from the end-to-end data. The specific goal is to estimate QoS parameters such as loss rates and delays at the link level. The reason for probing the network from the outside is that Internet service providers or other interested parties often do not have access to the internal nodes of the network (which may be owned by a third party). Nevertheless, they have to assess QoS of the links over which they are providing service. Active tomography offers a convenient approach by probing the network from nodes located on the periphery. The probing and data collection are done with dedicated instruments at the root node and receiver nodes. These packets can be sent to one receiver at a time (unicast transmission scheme) or to a specified subset of receivers (multicast scheme). Some networks have turned off the multicast scheme for security reasons. In this case, one sends unicast packets to several receivers spaced closely in time with the goal of trying to mimic the multicast scheme. What causes losses and delays of packets over the network? When a packet arrives at a node, it joins a queue of incoming packets. If the buffer is full, the packet is dropped, i.e., lost. Depending on the protocol, the packet may or may not be resent. Packets also encounter delays along the path, primarily due to the queueing process above. In the case of losses, the binary outcome Xk = 0 or 1 indicates whether the packet is lost (dropped) or not. In terms of the examples in Section 1, f(x1, . . . , xk) = k xk, and the end-to-end loss Y = k∈P(0,r) Xk = 1 if the packet transmitted along the path P(0, r) reached the receiver node r and zero otherwise. For delays, f(x1, . . . , xK) = k xk, and the end-to-end observation is Y = k∈P(0,r) Xk, the path-level delay. The physical topology of a network is usually complicated. But the logical topol- ogy with a single source node can often be represented as a tree. For example, the left panel of Figure 2 shows the physical topology of a subnetwork at the campus of the University of North Carolina at Chapel Hill. The right panel shows the cor- responding logical topology, which is a tree with a directed flow. We will revisit this network later in the paper. It is possible to deal with topologies with multiple sources, other kinds of transmission schemes (two-way flows), and so on. But for simplicity, we will restrict attention to the tree structures in this paper. Fig 2. Left panel: Schematic of the UNC network; Right panel: Logical topology of the UNC network.
- 40. 28 E. Lawrence, G. Michailidis and V. N. Nair 3. Literature review of loss and discrete delay inference Most of the results in the literature on active tomography have been developed under the assumption that the loss rates and delay distributions are temporally homogeneous and are independent across links. We will also use this framework. The assumption of temporal homogeneity is reasonable as the probing experiments are done within the order of minutes. The assumption of independence across links is less likely to hold. However, the nature of the dependence will vary from network to network, and it is difficult to obtain general results. 3.1. Design of probing experiments We noted in Example 1 that the link-level parameters are not identifiable under the unicast transmission scheme (sending probes to one receiver at a time). The multicast scheme, which sends packers to all the receivers in the network simultane- ously, addresses this problem for loss rates and, under some additional conditions, for delay distributions as well. However, this scheme has a number of drawbacks. It creates more traffic than necessary for estimating the link-level parameters. Also, the data generated are very high-dimensional. For example, in a binary symmetric tree with L layers, there are R = 2L − 1 receiver nodes. A multicast scheme for measuring loss rates results in a multinomial experiment with 2R possible outcomes. This is a large number even for moderately sized trees. The most important drawback, however, is that it is inflexible and does not allow investigation of subnetworks using different intensities and at different times. In practice, one may want to probe sensitive parts of the network as lightly as necessary to avoid disturbance. So there is a need for more flexible probing experiments. As pointed out in Example 4, this raises interesting issues on how to design the probing experiments. A class of flexible probing experiments, called flexicast experiments, were in- troduced and studied in Xi et al. [17] and Lawrence et al. [8]. This consists of a combination of schemes for different values of k with each scheme aimed at study- ing a subnetwork. However, each of the scheme by itself will not necessarily allow us to estimate the link-level parameters of that subnetwork. The data have to be combined across the various k-cast schemes to estimate the link-level parameters. To illustrate the ideas, consider the network on the right panel in Figure 1. The multicast scheme sends probes simultaneously to {2, 3, 6, 8, 9, 10, 11, 12, 13, 14, 15}. Two possible flexicast experiments are: (1) {2, 3, 6, 12, 13, 14, 8, 15, 9, 10, 11} and (2) {2, 3, 6, 12, 13, 14, 15, 8, 9, 10, 11}. The former consists of only bicast (two receiver nodes at a time) and unicast schemes. Intuitively, the latter scheme appears to more “efficient” but we will see shortly that it does not allow one to estimate all the link-level parameters. A full multicast scheme for this tree will result in 11-tuples or 11-dimensional data. The first flexicast experiment using pairs and singletons can cover the whole tree with five pairs and one singleton. The resulting data are considerably less complex in terms of processing and computations for inference. This advantage is particularly important for trees with many layers.
- 41. Inverse problems in network tomography 29 Of course, not all flexicast experiments will permit estimation of the link-level parameters. To discuss the technical issues associated with the identifiability prob- lem, consider first the notion of a splitting node. For a k-cast scheme, an internal node is a splitting node if the scheme splits at that node. For example, for the tree on the right panel of Figure 1, the bicast scheme {6, 12} splits at node 4. Xi et al. [17] showed that the following conditions are necessary and sufficient for iden- tifiability of link-level loss rates: (a) all receiver nodes are covered; and (b) every internal node in the tree is a splitting node for some k-cast scheme in the flexicast experiment. Lawrence et al. [8] studied the delay problem and showed that the same conditions are also necessary and sufficient for estimating delay distributions provided the distributions are discrete. The case where the delay distributions are not discrete is discussed in the next section. Consider again the flexicast schemes in equations (1) and (2) for the tree on the right panel in Figure 1. The first one based on a collection of bicast and unicast schemes satisfies the conditions. For the second one, none of the k-cast schemes split at node 4. There are many flexicast experiments that satisfy the identifiability requirements, and the choice among these has to be based on other criteria. Experiments based on just bicast and unicasts have minimal data complexity – just 1- and 2-dimensional outcomes. However, these provide information on just first and second-order de- pendencies and will be less efficient (in a statistical sense) to k-cast schemes with higher values of k. In particular, the full mulitcast scheme will be most efficient in this sense. So the overall choice of the flexicast experiment has to be a compromise between statistical efficiency and flexibility including the ability to adapt over time to accommodate changes in network conditions. 3.2. Inference for loss rates Inference for loss rates was first studied in Cáceres et al. [2] for the multicast scheme. A recent, up-to-date list of references can be found in Xi et al. [17] who developed MLEs based on the EM algorithm for flexicast experiments. We provide next a brief review of these results. Each k-cast scheme in a flexicast experiment is a k-dimensional multinomial ex- periment. Specifically, each outcome is of the form {Zr1 , . . . , Zrk } where Zrj = 1 or 0 depending on whether the probe reached receiver node rj or not. Let N(r1,...,rk) denote the number of outcomes corresponding to this event, and let γ(r1,...,rk) be the probability of this event. Then the log-likelihood for the k-cast scheme is propor- tional to γ(r1,...,rk) log(N(r1,...,rk)). The overall log-likelihood is just the sum of the log-likelihoods for these individual experiments. However, the γ(r1,...,rk) are compli- cated functions of αk, the link-level loss rates, so one has to use numerical methods to obtain the MLEs. The EM algorithm is a natural approach for computing the MLEs and has been used extensively in network tomography applications (see [3, 5, 16]). The structure of the EM-algorithm for general flexicast experiments was developed in Xi et al. [17]. While the E-step can be complex for arbitrary collections of k-cast schemes, it simplifies for flexicast experiments comprised of bicast and unicast schemes as seen below. Let sb be the splitting node for bicast pair b = ib, jb. Then, π(0, sb), π(sb, ib) and π(sb, jb), the three path probabilities for this bicast pair are products of the αk. Starting with an initial value α(0) let α(k) be the value after the k-th iteration. Then, we can write the (k + 1)-th iteration of the E-step as follows:
- 42. 30 E. Lawrence, G. Michailidis and V. N. Nair E-step: 1. For each bicast pair: (a) Use α(k) to obtain the updated path probabilities π(k) (0, sb), π(k) (sb, ib), π(k) (sb, jb) and γ b (k) 00 . (b) For each node ∈ P(0, sb) ∪ P(sb, ib) ∪ P(sb, jb), compute V (k+1) ,b = E α(k) [V| Nb], where Nb = {Nb 00, Nb 01, Nb 10, Nb 11} are the collected counts of the four possible outcomes, as follows. For node ∈ P(0, sb), V (k+1) ,b = Nb − Nb 00 1 − α (k) γ b (k) 00 . For link ∈ P(sb, ib), V (k+1) ,b = Nb − Nb 01 × 1 − α (k) 1 − π(k)(sb, ib) − Nb 00 (1 − α (k) )(1 − π(k) (0, jb)) γ b (k) 00 . For link ∈ P(sb, jb), V (k+1) ,b = Nb − Nb 10 × 1 − α (k) 1 − π(k)(sb, jb) − Nb 00 (1 − α (k) )(1 − π(k) (0, ib)) γ b (k) 00 . 2. Unicast schemes: Let node ∈ P(0, u) for a unicast transmission to receiver node u, and compute V (k+1) ,u = Nu − Nu 0 × 1 − α (k) 1 − π(k)(0, u) M-step: The (k + 1)-th update for the M-step is simply α (k+1) = b∈B V (k+1) ,b + u∈U V (k+1) ,u b∈B Nb + u∈U Nu where B is the set of bicast pairs that includes the node in its path and U is the set of all unicast schemes that includes node in its path. In our experience, the EM algorithm works reasonably well for small to moderate networks when used with a flexicast experiment that consists of a collection of bi- cast and unicast schemes. For large networks, however, it becomes computationally intractable. In on-going work, we are developing a class of fast estimation meth- ods based on least-squares methods and are studying their application to on-line monitoring of network performance. 3.3. Inference for discrete delay distributions For the delay problem, let Xk denote the (unobservable) delay on link k, and let the cumulative delay accumulated from the root node to the receiver node r be Yr = k∈P(0,r) Xk. Here P(0, r) denotes the path from node 0 to node r. The observed data are end-to-end delays consisting of Yr for all the receiver nodes. Most of the papers on delay inference assume a discrete delay distribution. Specif- ically, if q denotes the universal bin size, Xk ∈ {0, q, 2q, . . . , bq} is the discretized
- 43. Inverse problems in network tomography 31 delay on link k and bq is the maximum delay. Let αk(i) = P{Xk = iq}. The in- ference problem then reduces to estimating the parameters αk(i) for k ∈ E and i in {0, 1, . . . , b} using the end-to-end data Yr. Lo Presti et al. [10] developed a fast, heuristic algorithm for estimating the link delays. Liang and Yu [9] developed a pseudo-likelihood estimation method. Nonparametric maximum likelihood estima- tion under the above setting was investigated in Tsang et al. [13] and Lawrence et al. [8]. Shih and Hero [12] examined inference under mixture models. See also Zhang [18] for a more general discussion of the deconvolution problem. We discuss nonparametric MLE with discrete delays in more detail. Let αk = [αk(0), αk(1), . . . αk(b)] and let α = [ α 0, α 1, . . . , α |E|] . The observed end-to-end measurements consist of the number of times each possible outcome y was observed from the set of outcomes Yc for a given scheme c. Let Nc y denote these counts. These are distributed as multinomial random variables with corresponding path- level probabilities γc( y; α). So the log-likelihood is given by l( α; Y) = c∈C y∈Yc Nc y log[γc( y; α)]. This cannot be maximized easily, and one has to resort to numerical methods. Again, the EM algorithm is a reasonable technique for computing the MLEs. See [7] for multicast schemes and [8] for inference with flexicast experiments. How- ever, the complexity of the EM algorithm, in particular computing conditional expectations of the internal link delays for each bin, is prohibitive for all but fairly small-sized networks. To deal with larger networks, [8] developed a grafting method which fits “local” EMs to the subtrees defined by the k-cast schemes and then com- bines the estimates through a fixed point algorithm. This hybrid algorithm is fast and has reasonable statistical efficiency compared to the full MLE. For bicast schemes, the resulting algorithm has third-order polynomial complex- ity, a substantial improvement over the full bicast MLE. The heuristic algorithm in [10] is based on solving higher order polynomials and is much faster. However, it uses only part of the data and is quite inefficient. The pseudo-likelihood method of [9] uses only data from all pairs of probes in the multicast experiment. This is simi- lar in spirit to a flexicast experiment comprised of only bicast schemes, although in this setting the schemes would be independent. The computational performance of the pseudo-likelihood method is faster than the MLE based on the full multicast. It is comparable to doing a full EM based on data from all possible bicast schemes. This will still not scale up well to very large trees as it includes all possible bicasts which can involve a large number of schemes. Furthermore, using the full MLE combining the results across all schemes is computationally intensive. The flexicast experiments, on the other hand, are typically based on a much smaller number of schemes (eve if one restricts attention to bicasts). Further, the grafting algorithm is much faster for combining the results across the schemes. 3.4. Application to the UNC network We use a real example to demonstrate how the results from active tomography are used. The example deals with estimating the QoS of the campus network at the University of North Carolina at Chapel Hill and assessing its capabilities for Voice-Over-IP readiness. This network has 15 endpoints which were organized into the tree shown in Figure 2. Node 1 is the main campus router and it connects to the university gate- way. Nodes 2, 3, and 9 are also large routers responsible for different portions of
- 44. 32 E. Lawrence, G. Michailidis and V. N. Nair Fig 3. Probability of large delay on 3/7/2005. the campus. The accessible nodes are all located in dorms and other university buildings. The root node of the tree was Sitterson Hall which houses the computer science department. The network was probed in pairs using the following flexicast experiment: {4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18}. A single prob- ing session consisted of two passes through the collection of experiments sending about 500 probes to each pair in a single pass. The experiment was conducted over the course of several days in order to evaluate both the network and the methodol- ogy. We have collected extensive data but show only selected results for illustrative purposes. The data presented here were collected at 9:00 a.m., 12:00 p.m., 3:00 p.m., 6:00 p.m., and 9:00 p.m. on March 1 and 17 of 2005. March 17 was during spring break. For both days, we chose a bin size of q = .0001s to assess occurrences of large delays on the network. The large bin size also allowed us to use the full MLE to estimate the delay distributions. Figures 3 and 4 provide a picture of the probability of large delay (larger than a specified threshold) throughout the course of the day. From Figure 3, we see that many buildings (Venable, Davis, Rosenau, Smith, Greenlaw, and South) show a typical diurnal pattern. These buildings are either administrative or departmental building; so the majority of users follow a regular 9 to 5 schedule. Other buildings are either more uniform throughout the day or even more activity at night. Hinton, for example, is a large freshman dorm and thus the drop during the day and increase at night are expected as the residents return from classes and other activities in the evening. A comparison of Figures 3 and 4 shows the difference in dorm activity before
- 45. Inverse problems in network tomography 33 Fig 4. Probability of large delay on 3/17/2005. and during spring break. Everett, Old East, Hinton, and Craige are dorms. The data collected during spring break reveals almost no large delays in three out of four of these buildings. This is of course to be expected. The Hinton dorm is espe- cially interesting, since it experienced very little congestion over the break, but a significant increase to pre-break levels on the first day after the break (post-break results are not shown here). As a consequence of this study, it became clear that many of the building links require upgrades in order to support delay-sensitive applications such as VoIP. Some of the departmental and administration buildings (Smith and South) already have large delays even without additional VoIP traffic. 4. Parametric inference for delay distributions This section develops some new results on parametric inference for delay distribu- tions. We start with a framework that includes two components: a zero delay and a (non-zero) finite delay. Specifically, let Xk be the delay on link k, and suppose (3) Xk ∼ pkδ{0} + (1 − pk)F(x; θk). Here we assume that F(x; θk) does not give any mass to zero, for all k. So, a successful transmission (finite delay or no loss) experiences an empty queue (no delay) with probability pk and has some non-zero delay that is distributed according to a parametric distribution F(·) indexed by θk with probability 1 − pk.
- 46. 34 E. Lawrence, G. Michailidis and V. N. Nair Fig 5. Three-layer, binary tree 4.1. Identifiability The basic issue for delay distributions is the one posed in Example 2 in the intro- ductory section, viz., whether the parameters of a simple two-layer tree (left panel of Figure 1) are estimable from probes sent simultaneously to both receivers. If this holds, then the result extends readily to general flexicast experiments that satisfy the conditions in Section 3.1 (using the arguments in [8]). We discuss the details briefly. See also [4, 6] for a general discussion of identifiability issues. We consider two cases: Case 1: If pk 0 for all k, no additional assumptions on the distribution F(·) are needed. All the link-level delay parameters (pk and θk) are identifiable using flexicast experiments provided they satisfy the conditions in Section 3.1: a) every receiver node is covered and b) every internal node is a splitting node for some sub-experiment. To see this, consider the two-layer tree on the left panel of Figure 1. Condition on the subset of data with Y2,m = 0 and Y3,m 0 for probes m = 1, . . . , M. Now, Y2,m = 0 implies that both of the internal links X1,m and X2,m had zero delay, so Y3,m = X3,m. So we can use this subset of Y3,m to estimate F(x; θ3). A similar argument can be used to estimate F(x; θ2) using the subset of Y3,m 0 and Y2,m = 0. Once these two distributions are estimated, we can easily estimate F(x; θ1). Case 2: If pk = 0, then we need additional assumptions on the delay distributions F(x; θk). As we noted in Example 2, the means of the normal distributions are not identifiable. If the moments of order two and higher depend on the first mo- ment, they will provide additional information for estimating the parameters. One such example is when the variance is a function of the mean (as is the case with exponential, gamma, log-normal, and Weibull distributions). Example 5. We consider here a more general situation with the three-layer bi- nary, symmetric tree shown in Figure 5. Let the delay on link k be distributed Gamma(αk, βk). Suppose we use the flexicast probing experiment {4, 5, 5, 6, 6,
- 47. Inverse problems in network tomography 35 7}. The covariances yield the following moment equations: Cov(Y 5,6 5 , Y 5,6 6 ) = α1β2 1, Cov(Y 4,5 4 , Y 4,5 5 ) − Cov(Y 5,6 5 , Y 5,6 6 ) = α2β2 2, Cov(Y 6,7 6 , Y 6,7 7 ) − Cov(Y 5,6 5 , Y 5,6 6 ) = α3β2 3. Let E(Yr) = νr. We also get the following equations based upon third moments: E(Y 5,6 5 − ν5)2 (Y 5,6 6 − ν6) = 2α1β3 1, E(Y 4,5 4 − ν4)2 (Y 4,5 5 − ν5) − E(Y 5,6 5 − ν5)2 (Y 5,6 6 − ν6) = 2α2β3 2, E(Y 6,7 6 − ν6)2 (Y 6,7 7 − ν7) − E(Y 5,6 5 − ν5)2 (Y 5,6 6 − ν6) = 2α3β3 3. The corresponding sample moments can be used to estimate the terms on the left. Then, estimators for α1, β1, α2, β2, α3, and β3 can be obtained by rearranging the above equations. The parameters for the receiver links can be estimated with just the first mo- ments. For example, the equations for link 4 are: E(Y4) = α1β1 + α2β2 + α4β4, Var(Y4) = α1β2 1 + α2β2 2 + α4β2 4. The unknown parameters are easily obtained from the observed values on the left and the estimated parameters on the right. 4.2. Maximum likelihood estimation It turns out that pk, the probability of zero delay, can be estimated using methods analogous to those for loss rate discussed in Section 3.2. Recall that a zero delay will be observed at the receiver node if and only if there is zero delay at every link. On the other hand, a non-zero delay at the receiver link may include zero delays at some links, so we have to “recover” this information from the aggregate level data. But this is equivalent to the problem with of losses. A packet received at the receiver node implies “success” at all the links. A packet not received at the receiver node involves a combination of successes and losses, with at least one loss. Thus, we can use the data with zero-delays and positive delays in an analogous manner to estimate the zero-delay probabilities pk. To simplify matters, therefore, we will focus on parametric estimation of Fk(x; θk) assuming that pk = 0. Let us consider some simple examples with the two-layer tree with two receivers in the left panel of Figure 1 and with exponential and gamma distributions for delays. The gamma family is closed under convolution if the scale parameters are the same, so the distribution of the end-to-end delays belong to the same family as the link-level delays. Even for these simple cases, we will see that the MLE computations are intractable. a) Exponential Distributions: Suppose the delay distribution on each link is exponential with parameter λk. Further, we send N probes to both receivers 2, 3
- 48. 36 E. Lawrence, G. Michailidis and V. N. Nair simultaneously. The log-likelihood function is l( λ; Y) = N log(λ1) + N log(λ2) + N log(λ3) − N log(λ1 − λ2 − λ3) (4) − λ2 N i=1 yi,2 − λ3 N i=1 yi,3 + N i=1 log[1 − exp{−(λ1 − λ2 − λ3) min(yi,2, yi,3)}] There is no analytic solution to maximize this equation over λ, so one would have to use an iterative technique, such as EM or Newton-Raphson, to find the MLEs even in this simple case. We examine the details for the EM-algorithm. The exponential distribution is a member of the exponential family, so the (unobserved) sufficient statistics are the total link-level delays n i=1 Xi,1, n i=1 Xi,2, and n i=1 Xi,3. Since Xi,2 = Yi,2 −Xi,1 and Xi,3 = Yi,3 − Xi,1, we need to compute only the conditional expected values of n i=1 Xi,1 in the E-step. The conditional distribution [X1|Y2 = a, Y3 = b] has density (5) g(x) = exp{−(λ1 − λ2 − λ3)x} C(a, b; λ1, λ2, λ3) , 0 x a ∧ b, where a ∧ b = min(a, b) and the constant of proportionality is C(a, b; λ1, λ2, λ3) = 1 − exp{−(λ1 − λ2 − λ3)(a ∧ b)} λ1 − λ2 − λ3 . Now a∧b 0 xg(x)dx is an incomplete gamma function and one can compute the expected value n i=1 E(Xi,1|Yi,2, Yi,3) as a ratio of the incomplete gamma function and the constant C(a, b; λ1, λ2, λ3). Thus, the MLEs of the λk can be computed without too much trouble in this simple two-layer binary case. How well does this extend to more general cases? Suppose we have a three layer binary tree (Figure 5), and we use bicast schemes 4, 5, 6, 7, and 5, 6. Consider the scheme 4, 5 which splits at node 2. We can try and mimic the computations for the two-layer tree above. However, we have to consider the combined path P(0,2) whose delay is the sum of delays for links 1 and 2. The exponential distribution is not closed under convolution, so the distribution is now more complex. The details for more general trees will depend on the number of links involved before-and-after the splitting node. The problem is even more complex for multicast schemes with multiple splitting nodes. We see that the MLE computations are complicated even for simple exponential distributions. b) Gamma Distributions: Gamma distributions with same scale parameter are closed under convolution, i.e., the path delays which are sums of link-level delays are still gamma. Specifically, let Xk ∼ Gamma(αk, β) and independent across k for k ∈ E. We start with the simple two-layer binary tree. Then, the likelihood function
- 49. Inverse problems in network tomography 37 of the observed data is L(data) = n i=1 yi,2∧yi,3 0 f1(x)f2(yi,2 − x)f3(yi,3 − x)dx = n i=1 yi,2∧yi,3 0 1 Γ(α1) 1 βα1 xα1−1 exp{− x β } × 1 Γ(α2) 1 βα2 (yi,2 − x)α2−1 exp{− yi,2 − x β } × 1 Γ(α3) 1 βα3 (yi,3 − x)α3−1 exp{− yi,3 − x β } dx = n i=1 1 Γ(α1)Γ(α2)Γ(α3) 1 βα1+α2+α3 exp{− 1 β (yi,2 + yi,3)} × yi,2∧yi,3 0 xα1−1 (yi,2 − x)α2−1 (yi,3 − x)α3−1 exp{ xi β }dx . As before, the MLEs will have to be obtained numerically. Let us consider the details of the EM-algorithm. The Gamma distribution is a member of the exponential family with sufficient statistics X and log(X). For the two-layer tree, we need to compute just the conditional expectation of X1 and log(X1), the unknown delays on the first link. The conditional distribution [X1|Y1 = a, Y2 = b] is now given by (6) g(x) = xα1−1 (a − x)α2−1 (b − x)α3−1 exp{x β } C(a, b; α1, α2, α3, β) , 0 x a ∧ b, where the proportionality constant is C(a, b; α1, α2, α3, β) = a∧b 0 xα1−1 (a − x)α2−1 (b − x)α3−1 exp{ x β }dx. This can be used to compute E[X1|Y1, Y2] and E[log(X1)|Y1, Y2] numerically. Note that E[X1|Y1, Y2] = C(Y1, Y2; α1 + 1, α2, α3, β) C(Y1, Y2; α1, α2, α3, β) . How well does this extend to trees with more than two layers? It turns out that the full MLE is still not feasible. However, a combination of “local” MLEs and a grafting idea (along the lines of [8]) is feasible. Consider the 3-layer tree in Figure 5. Suppose we use a flexicast experiment with 3 bicasts 4, 5, 6, 7, and 5, 6. The bicast scheme 4, 5 splits at node 2. So we can combine links 1 and 2 into a single link and use the previous results for the two-layer tree to get estimates for this subtree. Note that the delay distribution for the combined links 1 and 2 is Γ(α1 +α2, β). So we can get “local” MLEs for α1 +α2, α4, α5 and β from the bicast experiment 4, 5. Using a similar argument, we can get estimates for α1 + α3, α6, α7 and β from the bicast scheme 6, 7 and estimates for α1, α2 + α5, α3 + α7 and β from the bicast scheme 5, 6. Now we can use one of several methods to combine these estimates to get a unique set of estimates for all of the αk and β. Possible methods include ordinary or weighted LS. We do not pursue this strategy here as the specifics work only for special cases. The main message here is that it is not easy to compute the full MLE even in very simple cases, and the problem becomes completely intractable as the size of the tree and number of children in the links grow.
- 50. 38 E. Lawrence, G. Michailidis and V. N. Nair 4.3. Method-of-moments estimation We discuss the use of method of moments which estimates the parameters by match- ing the population moments to the sample moments using some appropriate loss function. General losses are possible, but squared error loss leads to more tractable optimization and the large-sample properties are easy to establish. Let H = {1, . . . , m} be the index set of the probes used in the probing exper- iment. Denote the observed data Yr(1), . . . , Yr(m) as Yr(H). Let Mj i (H) be the observed i-th moment for the j-th scheme based on the probes in H. Let Mj i (θ) be the functional form of the i-th moment from the j-th probing scheme. For example, for the two-layer tree in Figure 1 with Gamma(αk, βk) distributions on each link, we get the following relationships: E(Y2) = α1β1 + α2β2, E(Y3) = α1β1 + α3β3, Cov(Y2, Y3) = α1β2 1, E(Y2 − ν2)2 (Y3 − ν3) = 2α1β3 1, Var(Y2) = α1β2 1 + α2β2 2, Var(Y3) = α1β2 1 + α3β2 3. We can now estimate the parameters by minimizing the squared error loss Q(θ; M(H)) = m j=1 i Mj i (H) − Mj i (θ) 2 . This is a special case of the nonlinear least squares problem and can be solved using iterative methods such as the Gauss-Newton procedure (see [1] for example). After rewriting the loss function as a single sum over all the moments, we consider the derivatives (7) ∂Q(θ; M(H)) ∂θj = −2 i [Mi(H) − Mi(θ)] ∂Mi(θ) ∂θj . These can be expressed in matrix form as ∂Q(θ; M(H)) ∂θ = D [M(H) − M(θ)], where Di,j = ∂Mi(θ) ∂θj . The moments at the true value can be expanded using a Tay- lor series expansion around an initial guess θ(0) as M(θ0) ≈ M(θ(0) )+D(θ0 −θ(0) ). Computing the residuals and replacing the true value with the observed moments gives an updating scheme based on solving a linear system. Thus at iteration q, we have the following linear system. M(H) − M(θ(q) ) = Dβ. Solving this, we get the next iteration as θ(q+1) = θ(q) + β̂. In general, each iteration should be closer to the minimizer. However, there can be situations where the step increases the sum of squares. To avoid this, we recommend the modified Gauss-Newton in which the next iteration is given by θ(q+1) = θ(q) + rβ̂ where 0 r ≤ 1. This fraction can be chosen adaptively at each
- 51. Inverse problems in network tomography 39 step. If the full step reduces the sum of squares, then it is taken. Otherwise, we can set r = .5. If the half step fails to reduce the sum of squares, then it can be halved again. This guarantees that the loss function is reduced with every step and gives convergence to a stationarity point. Examination of the derivatives will indicate if the point is a minimum. The algorithm has useful complexity properties in terms of both memory and computation. Since the estimation is based only on the moments, these values are all that need to be stored. This is a vast improvement over algorithms that require all of the data or the counts of the binned data. Further, the efficient implementation of the algorithm, involving a QR factorization and one matrix inversion, gives compu- tational complexity of O(m3 ) where m is the number of required moments. Again, this is a large improvement over other methods that have exponential complexity. Further improvement is achieved in many cases using sparse matrix techniques. These two points make the approach ideal for application requiring real-time esti- mates. The ordinary non-linear least-squares (OLS) method of moment (MOM) esti- mators can be inefficient as the different moments are correlated and have unequal variances. Since these can often be computed and estimated easily, one can use generalized least-squares (GLS) to improve the efficiency. A limited comparison of the efficiencies is given in the next subsection. It is easy to show that the method-of-moment estimators based on OLS or GLS are consistent and asymptotically normal as the sample sizes (numbers of probes) increase on a given network. The large-sample distribution can be used to compute approximate confidence regions and hypothesis tests which are useful in monitoring applications. 4.4. Relative efficiency of the method-of-moments: a limited study We conducted a small simulation study to assess the performance of the MOM estimators versus the MLE. This was done on a two-layer binary tree (left panel of Figure 1) for exponential distributions. This is one of the few instances when it is practical to construct the MLE. We used two MOM estimators: the OLS and the GLS schemes (described above). The GLS methods used a weighting scheme based on an empirical estimate of the covariance of the observed moments. Relative efficiency is defined as the ratio of the variance of the MLE to the variance of the estimator of interest. We considered two cases: a) each link has the same mean; i.e., 1/λk = 1/2, k = 1, 2, 3, and b) each link has its own mean, 1/λ1 = 1/2, 1/λ2 = 1/4, and 1/λ3 = 1/6, respectively. For both scenarios, 1000 data sets of size 3000 were generated. Boxplots of the three sets of estimators are shown in Figures 6 and 7. The procedures appear to be unbiased. When the means are the same, the relative efficiencies of the OLS MOM are 1.72, 1.33, 1.41 and the relative efficiencies of the GLS MOM are 1.12, 1.11, and 1.12. When the means are different, the relative efficiencies for the OLS are 2.43, 5.09, and 9.13 and the relative efficiencies for the GLS are 1.07, 1.24, and 1.34. In this example, the GLS MOM appears to be quite efficient compared to the MLE. However, a much more extensive study is clearly needed to quantify the performance of the MOM estimators.
- 52. 40 E. Lawrence, G. Michailidis and V. N. Nair Fig 6. Boxplots comparing the MLE with MOM: Case 1 – All means are equal. Fig 7. Boxplots comparing the MLE with MOM: Case 2 – The three means are different.
- 53. Inverse problems in network tomography 41 4.5. Analysis using the NS-2 simulator We now describe a study of the MOM estimators in a simulated network environ- ment. The ns-2 package is a discrete event simulator that allows one to generate network traffic and transmit packets using various network transmission protocols, such as TCP, UDP, ([15]) over wired or wireless links. The simulator allows the underlying link delays to exhibit both spatial and temporal dependence with corre- lation between sister links (children of the same parent, i.e. 4, 5, and 6.) around .25 and autocorrelation about .2 on all of the links. Thus, we can study the performance of the active tomography methods under more realistic scenarios. We used the topology shown in Figure 8 with a multicast transmission scheme. The capacity of all links was set to the same size (100 Mbits/sec), with 11 sources (10 TCP and one UDP) generating background traffic. The UDP source sent 210 byte long packets at a rate of 64 kilobits per second with burst times exponentially distributed with mean .5s, while the TCP sources sent 1,000 byte long packets every .02s. The main difference between these two transmission protocols is that UDP transmits packets at a constant rate while TCP sources linearly increase their transmission rate to the set maximum and halve it every time a loss is recorded. The length of the simulation was 300 seconds, with probe packets 40 bytes long injected into the network every 10 milliseconds for a total of about 3,000. Finally, the buffer size of the queue at each node (before packets are dropped and losses recorded) was set to 50 packets. We studied only the continuous component of the delay distribution, i.e., the portion of the path-level data that contain zero or infinite delays was removed. The traffic-generating scenario described above resulted in approximately uniform waiting times in the queue (see Figure 9). This is somewhat unrealistic in real network situations where traffic tends to be fairly bursty [11], but it provides a simple scenario for our purposes. Estimating the unknown parameters for this model is equivalent to estimating the maximum waiting time for a random packet. Figure 10 shows quantile-quantile plots using simulated values from the fitted distributions versus the observed ns-2 delays for both the links and paths. Specifi- cally, we estimated the parameters for the uniform distributions using the moment estimation procedure and then generated data based on these parameter values. The fitted values were: b̂ = [.89, .79, .53, 1.10, 1.09, 1.13]. The estimation procedure does quite well on all of the links except for the interior link 3. The algorithm seems Fig 8. Portion of the UNC network used in the ns-2 simulaton scenario.
- 54. 42 E. Lawrence, G. Michailidis and V. N. Nair Fig 9. Histogram of link delays for the ns-2 simulation. to compensate for this under-estimation (about 40%) by slightly overestimating the parameters for each of the descendants of link 3, as evidenced by the closely matched quantiles for the end-to-end data. This error is probably the result of several fac- tors. First, link 3 deviates the most from the uniform distribution with the last bin in Figure 9 being too thin. Secondly, the algorithm appears to be moderately affected by the violations of the independence assumptions, particularly the spatial dependence among the children of link 3. This could likely be somewhat relieved by using a larger sample size and accounting for the empty queue probabilities. Nevertheless, the estimation performs well overall. 5. Summary There are a number of other interesting problems that arise in active network to- mography. There are usually multiple source nodes, which raises the issue of how to optimally design flexicast experiments for the various sources. We have also assumed that the logical topology of the tree is known. However, only partial knowledge of the network topology is typically available, and one would be interested in using the path level information to simultaneously discover the topology and estimate the parameters of interest. The topology discovery problem is computationally dif- ficult (NP-hard), but methods using a Bayesian formulation as well as those based on clustering ideas have been proposed in the literature (see [3] and references therein). Active tomography techniques are useful for monitoring network quality of ser- vice over time. However, this application requires that path measurements are col- lected sequentially over time and appropriately combined. The probing intensity, the type of control charts to be used for monitoring purposes, and the use of path- vs link- information are topics of current research.
- 55. Other documents randomly have different content
- 56. TOP OF THE MARSAN PAVILION, LOUVRE. remain together very long. The Emperor Napoleon was, after the catastrophe of Sedan, to be replaced by the Republican Government of the 4th of September, which was soon to give way to the Commune, under whose abominable rule so many fine buildings, with the Palace of the Tuileries among them, were wantonly sacrificed, and in a spirit of blind hatred burnt down. The conflagration lighted by the Communists had left standing and comparatively uninjured the outer walls, and therefore the general outline of the palace. But these were calmly pulled down by the “moderate” Republicans, less through considerations of art than from political prejudice. The Louvre subsists in its entirety, and in virtue of its magnificent collection of pictures, constantly enriched through sums voted during the last hundred years by National Assemblies, it has come to be looked upon as public property. The Tuileries, however, was a palace to the last; and the destruction of this palace, which the communards had only partially accomplished, was effectually completed by the “moderate” Republic established on the ruins of its immediate predecessor. Interesting as the Louvre may be by its ancient history, the old palace is above all famous in the present day for its admirable picture {201} gallery, first thrown open to the public in the darkest, most sanguinary days of the French Revolution. The modern collection was formed by Francis I., who, during his Italian campaigns, had acquired a taste for Italian art, and who not only invited celebrated Italian artists to his court, but gave princely orders to those who, like Raphael and Michel Angelo, were unable to visit France in person. He collected not only pictures, but art works, and especially antiquities of all kinds—statues, bronzes, medals, cameos, vases, and cups. Primatice alone brought to him from Italy 124 ancient statues and a large
- 57. number of busts. These treasures were collected at Fontainebleau, and a description of them was published long afterwards by Father Dan, who, in his “Wonders of Fontainebleau” (1692), names forty-seven pictures by the greatest masters, nearly all of which had been acquired by Francis I. It was not, indeed, until the reign of Louis XIII. that any important additions were made to Francis I.’s original collection. Among the pictures cited by Father Dan may in particular be mentioned two by Andrea del Sarto, one by Fra Bartolommeo, one by Bordone, four by Leonardo da Vinci, one by Michel Angelo (the Leda, afterwards destroyed), three by Perugino, two by Primatice, four by Raphael, three by Sebastian del Piombo, and one by Titian. THE MARSAN AND FLORA PAVILIONS, LOUVRE, FROM THE PONT ROYAL. The royal gallery was considerably augmented under the reign of Louis XIV. At his accession it included only 200 pictures. At his death the number had been increased to 2,000. Most of the new acquisitions were due to the Minister Colbert, who spared neither money nor pains to enrich the royal gallery, the direction and preservation of which was entrusted to the painter Lebrun. A banker, Jabach of Cologne, resident at Paris, had purchased a large portion of art treasures collected by King Charles I., and brought them over to Paris. He had bought many pictures, moreover, in various parts of the
- 58. Continent. Ruined at last by his passion for the fine arts, he sold a portion of his collection to Cardinal Mazarin, and another portion, composed chiefly of drawings, to the king. On Mazarin’s death, Colbert bought for Louis XIV. all the works of art left by that Minister, including 546 original pictures, 92 copies, 130 statues, and 196 busts. Louis XIV. placed his collection in the Louvre, and his first visit to the palace after the installation of the pictures is thus described in Le Mercure Galant of December, 1681:— “On Friday, the 5th day of the month, the king came to the Louvre to see his collection of pictures, which have been placed in a new series {202} of rooms by the side of the superb gallery known as the Apollo Gallery. The gold which glitters on all sides is the least brilliant of its adornments. What is called ‘the cabinet of his Majesty’s pictures’ occupies seven large and lofty halls, some of which are more than 50 feet long. There are, moreover, four additional rooms for the collection in the old Hôtel de Grammont adjoining the Louvre. So many pictures in so many rooms make the entire number appear almost infinite. The walls of the highest rooms are covered with pictures up to the ceiling. The following will give some idea of the number of pictures, by the greatest masters, contained in the eleven rooms:—There are sixteen by Raphael, six by Correggio, five by Giulio Romano, ten by Leonardo da Vinci, eight by Giorgione, twenty-three by Titian, sixteen by Carraccio, eight by Domenichino, twelve by Guido, six by Tintoretto, eighteen by Paul Veronese, fourteen by Van Dyck, seventeen by Poussin, and six by M. Lebrun, among whose works there are some (the battles of Alexander) which are 40 feet long. Besides these pictures there are a quantity of others by Rubens, Albano, Antonio Moro, and other masters of equal renown. Apart from the pictures, there are in the old Hôtel de Grammont many groups of figures and low reliefs in bronze and ivory.” The royal visit, as described by the writer in La Mercure Galant, was followed by the dispersion of the collection. Louis XIV. was so pleased by the wonderful sight that he ordered a number of the pictures to be removed to Versailles, where, according to the Mercure, there were already twenty-six pictures by the first masters; and so long as Versailles was the royal residence the greater part of the king’s collection was lost to the public, and served only to furnish the rooms, except, indeed, when the pictures had fallen to the ground and lay there covered with dust. Under the reign of Louis XIV. a critic whose name is worth preserving, Lafont de St. Yenne, complained that so many beautiful works were allowed to lie heaped up
- 59. together and buried in “the obscure prison of Versailles,” and demanded that all these treasures, “immense but unknown,” should be “arranged in becoming order and preserved in the best condition” in a gallery built expressly for their reception in the Louvre, where they would be “exhibited to the admiration and joy of the French or the curiosity of foreigners, or finally to the study and emulation of our young scholars.” The author of these judicious suggestions got into trouble as a pamphleteer; but four years afterwards, in 1750, Louis XIV. allowed the masterpieces previously stowed away in the apartments of the household at Versailles to be taken to Paris and submitted to the admiration of painters and lovers of painting. The Marquis de Marigny, Director of Royal Buildings, ordered Bailly, keeper of the king’s pictures, to arrange the collection in the apartments which had been occupied at the Luxembourg by the Queen of Spain. The “cabinet,” composed of 110 pictures, was opened for the first time October 14th, 1750, and the public was admitted twice every week, on Wednesdays and Saturdays. The pictures dedicated by Rubens to Marie de Médicis were on view the same days, and during the same hours. Until the reign of Louis XVI. the royal pictures, the number of which had been increased by the purchase of many examples of the Flemish school, continued to be divided into two principal sections, one placed in the Luxembourg, and visible twice a week to the public, the other kept out of sight in the palace of Versailles. The Louvre contained the “king’s cabinet of drawings,” to the number of about 10,000. The Apollo Gallery, which served as studio to six students patronised by the king, contained “The Battles of Alexander,” and some other pictures by Lebrun, Mignard, and Rigaud. In 1775, under Louis XVI., Count d’Angiviller succeeded the Marquis de Marigny, and going a step beyond him, formed the project of collecting everything of value that the Crown possessed in the way of painting and sculpture. Contemporary writers applauded this idea, which was attributed by some to M. de la Condamine. All, however, that came of the new proposal was that instead of pictures being brought from Versailles to Paris, the Louvre collection was transferred to Versailles. “It was necessary,” writes M. Viardot, “that a new sovereign—the nation —should come into power for all these immortal works rescued from the royal catacombs to be restored to daylight and to life. Who could believe, without authentic proofs, without official documents, at what epoch this
- 60. great sanctuary, this pantheon, this universal temple consecrated to all the gods of art, was thrown open to the public? It was in the middle of one of the crises of the Revolution in that dreadful year 1793, so full of agitation, suffering, and horror, when France was struggling with the last energy of despair against her enemies within and without; it was at this supreme moment that the {203} National Convention, founding on the ruins of the country a new and rejuvenated land, ordered the formation of a national art collection.” A step in this direction had already been taken in 1791, when it was decreed that the artistic treasures of the nation should be brought together at the Louvre. The year following, August 14th, 1792, the Legislative Assembly appointed a commission for collecting the statues and pictures distributed among the various royal residences; and on the 18th of October in the same year, Roland, Minister of the Interior, wrote to the celebrated painter David, who was a member of the Convention, to communicate to him the plan of the new establishment. Finally, a decree of July 27th, 1793, ordered the opening of the “Museum of the Republic,” and at the same time set forth that the “marble statues, vases, and valuable pieces of furniture placed in the houses formerly known as royal, shall be transported to the Louvre, and that the sum of 100,000 francs shall be placed annually at the disposition of the Minister of the Interior to purchase at private sales such pictures and statues as it becomes the Republic not to let pass into foreign hands, and which will be placed in the Museum of the Louvre.” It should not be forgotten that France was then at war with all the German Powers, and threatened by all the Powers of Europe. Crushed by military expenditure, the Republic had yet money to spare for the purchase of works of art. The French Museum, as the Louvre collection was first called, received afterwards the name of Central Museum of the Arts; and it was first opened to the public on the 8th of November, 1793. The next decree in connection with the fine arts ordered that a number of pictures and statues formerly belonging to the palace of Versailles, and which the inhabitants of Versailles were detaining as their property, should be placed in the Louvre. The old palace was still inhabited by a number of artists and their families. David had his studio there, and most of the painters who had made for themselves a tolerable reputation had apartments in the Louvre. It was reserved for Napoleon to turn them all out, and to give to the Louvre the character which it has since preserved—that of a national palace of art treasures.
- 61. The galleries of the Louvre profited greatly by the Napoleonic wars. All continental Europe was laid under contribution by the victorious French armies, but especially Italy and Spain. The stolen pictures formed the best part of what was now called the Musée Napoléon. Though not surreptitiously obtained they had been acquired in virtue of conventions imposed on a conquered people. Thus pictures from the galleries of Parma, Piacenza, Milan, Cremona, Modena, and Bologna, were made over to France by the armistices of Parma, Bologna, and Tolentino. The public was admitted to view the conquered treasures on the 6th of February, 1798. Some months afterwards masterpieces from Verona, Mantua, Pesaro, Loretto, and Rome were added to the marvellous collections; which on the 19th of March, 1800, was further augmented by drafts of pictures from Florence and Turin. In 1807 France received the artistic spoils of Germany and Holland. Among the famous works of art which France at this time possessed, and which were all on exhibition at the Louvre, may be mentioned “The Belvedere Apollo,” “The Laocoon,” “The Medicean Venus,” “The Wrestlers,” “The Transformation” and “The Spasimo”; Domenichino’s “Communion of St. Jerome,” Tintoretto’s “Miracle of St. Mark,” Paul Veronese’s four “Last Suppers,” and Titian’s “Assumption”; Correggio’s “St. Jerome” and Guercino’s “St. Petronilla”; “The Lances” of Velasquez, and the “St. Elizabeth” of Murillo; Rubens’ “Descent from the Cross,” and Rembrandt’s “Night Patrol.” The French say with some justice that many of these works by being sent to the Louvre were saved from destruction. Many of them, too, though falling into decay, were restored with the greatest care; and some were transferred with success from worm-eaten panels to canvas, thus receiving new brilliancy and a new life. When Paris was occupied by the allies in 1814, the art treasures of which so many European countries had been despoiled were left in the possession of the French, who may be said on this occasion to have been magnanimously treated. The object, indeed, of the allies was not to weaken nor to humiliate France as a nation, but simply to restore Louis XVIII. to the throne of his ancestors. In 1815, after the return from Elba and the Waterloo campaign, it was determined to treat France with a certain severity. She was deprived of the Rhine provinces for the benefit of Prussia, while Milan and Venice were placed in the hands of Austria, so that both from the Italian and from the
- 62. German side France might be held in check. The artistic plunder which France had collected from so many quarters was at the same time {204} given back to the countries from which it had been taken. French statesmen protested that the pictures and statues brought to Paris from so many foreign picture galleries belonged to France in virtue of formal treaties and conventions; Louis XVIII. himself declined to sanction the restoration of the captured pictures and statues. Denon, Director-General of Museums, resisted even when threatened with imprisonment in a Prussian fortress; and he made the foreign commissaries sign a declaration to the effect that in giving up the works claimed he yielded only to force. The so-called spoliation of the Louvre was at last effected. The pictures and statues, that is to say, which had been seized by victorious France, were from vanquished France taken back and replaced in the museums to which they had originally belonged. Since the fall of the First Empire the Louvre has acquired but few masterpieces from abroad. Italy now guards her art treasures with a jealous hand; and there are few countries where the masterpieces of antiquity can be purchased except when some private gallery is broken up through the bankruptcy or death of the owner. Under the new monarchy the beautiful though armless Venus of Milo was brought to France; and under the Second Empire “The Conception” of Murillo was purchased for 615,000 francs. The Third Republic, under the presidency of M. Thiers, spite of its difficulties in connection with the crushing war indemnity, paid 206,000 francs for a fresco by Raphael. The regular annual allowance to the Minister of Fine Arts for the purchase of pictures is now 100,000 francs a year. Meanwhile, the Louvre collection has been constantly augmented by pictures transferred to the more classical museum from the gallery of pictures by living artists in the Luxembourg. The pictures exhibited at the Louvre are arranged on a system which leaves nothing to be desired. The supreme masterpieces of the collection are all together, without reference to school, nationality, or period, in a large square room known as the Salon Carré. In the other rooms the pictures are arranged historically. The principal entrance to the picture galleries of the Louvre is in the Pavilion Molière, opposite the square of the Carrousel. After passing a spacious vestibule, where mouldings of Trajan’s Column and a fine collection of antique busts may be seen, the visitor ascends a staircase
- 63. adorned with Etruscan works in terra-cotta and reaches the round hall or cupola of the magnificent Apollo Gallery, decorated with wall paintings and painted ceilings by the courtly Lebrun of Louis XIV.’s time and the vigorous imaginative Eugène Delacroix of our own. What can be more admirable than Delacroix’s “Nymph,” at whose feet crouches a panther? “Behold this work,” writes Théophile Gautier, “and you will see that for colour France has no longer any reason for envying Italy, Flanders, or Spain. Delacroix, in this great page, in which the energy of his talent is freely displayed, shows a knowledge of decorative art which has never been surpassed. Impossible while never departing from his own genius to be more in harmony with the style of the gallery and of the epoch. One might here call him a florid romantic Lebrun.” The Apollo Gallery leads to the before-mentioned Salon Carré, where Paul Veronese’s “Marriage of Cana” at once attracts attention, not only by its immense proportions, but also and above all by the richness of the colouring and the beauty of the composition. Here, too, is the portrait by Leonardo da Vinci, known in France as “La Joconde”; “a miracle of painting,” says Gautier, who has made it the subject of one of his most remarkable criticisms. “‘La Joconde,’ sphinx of beauty,” he exclaims, “smiling so mysteriously in the frame of Leonardo da Vinci, and apparently proposing to the admiration of centuries an enigma which they have not yet solved, an invincible attraction still brings me back towards you. Who, indeed, has not remained for long hours before that head, bathed in the half-tones of twilight, enveloped in transparency; whose features, melodiously drowned in a violet vapour, seem the creation of some dream through the black gauze of sleep? From what planet has fallen in the midst of an azure landscape this strange being whose gaze promises unheard-of delights, whose experience is so divinely ironical? Leonardo impresses on his faces such a stamp of superiority that one feels troubled in their presence. The partial shadow of their deep eyes hides secrets forbidden to the profane; and the inflexions of their mocking lips are worthy of gods who know everything and calmly despise the vulgarities of man. What disturbing fixity, what superhuman sardonicism in these sombre pupils, in these lips undulating like the bow of Love after he has shot his dart. La Joconde would seem to be the Isis of some cryptic religion, who, thinking herself alone, draws aside {205} the folds of her veil, even though the imprudent man who might surprise her should go mad and die. Never did feminine ideal clothe itself in more
- 64. irresistibly seductive forms. Be sure that if Don Juan had met Monna Lisa he would have spared himself the trouble of writing in his catalogue the names of 3,000 women. He would have embraced one, and the wings of his desire would have refused to carry him further. They would have melted and lost their feathers beneath the black sun of these {206} eyes.” THE RICHELIEU PAVILION. Leonardo da Vinci is said to have been four years painting this portrait, which he could not make up his mind to leave and which he never looked upon as finished. During the sittings musicians played choice pieces in order
- 65. to entertain the beautiful model, and to prevent her charming features from assuming an expression of wearisomeness or fatigue. Raphael is represented in the Salon Carré by “St. Michael and the Demon,” painted on a panel framed in ebony. This admirable work is signed not in the corner of the picture, but on the edge of the archangel’s dress. “Raphaël Urbinas pingebat, M.D. XVIII.” runs the inscription, which Raphael seems to have wished to make inseparable from the work. Among the other pictures of Raphael chosen for places of honour in the Square Room are “The Holy Family,” which originally belonged to Francis I., and the virgin known as “La Belle Jardinière. Among the other masterpieces contained in the Salon Carré may be mentioned Correggio’s “Antiope,” Titian’s “Christ in the Tomb,” Giorgione’s “Country Concert,” Guido’s “Rape of Dejanira,” Rembrandt’s “Carpenter’s Family,” Van Ostade’s “Schoolmaster,” Gerard Douw’s “Dropsical Woman,” Rubens’ Portrait of his Wife, a “Charles I.” by Van Dyck, and Murillo’s “Conception of the Virgin.” This last-named work, as already mentioned, was purchased under the Second Empire for upwards of 600,000 francs. It formed part of a valuable collection of Spanish pictures belonging to Marshal Soult, and had been acquired by that commander under peculiar circumstances during the Peninsular War. A certain monk had been sentenced to death as a spy. Two monks from the same monastery waited upon the marshal to solicit their brother’s forgiveness. Soult was obdurate, until at last Murillo’s wonderful picture was placed before him. The picture was forwarded to France, and the too patriotic monk set free. Among the selected works by Italian, Dutch, Flemish, and Spanish painters are to be found a few by French artists—for example, the “Diogenes” of Poussin and the “Richelieu” of Philippe de Champagne; but not one work by an English hand. Nor in the famous Salon Carré of the Louvre is a single landscape to be found. The Tuileries, before incendiarism under the Commune rendered it a very imperfect building, had as a palace led a very imperfect life. Catherine de Médicis had ordered the destruction of the Palais des Tournelles, where, by a fatal accident Montgomery had pierced the eye and brain of Henri II. in the celebrated tournament, and had gone to live with her children at the Louvre. These children were Francis II., the husband of Marie Stuart; Charles IX., whose memory, like that of his mother, is indelibly associated with the massacre of St. Bartholomew; Henri III., who for his sins was elected King
- 66. of Poland; and Francis d’Anjou, who gained the famous battle of Jarnac, and who on his death was succeeded by Henri IV., first King of France and of Navarre. The ancient fortress of the Louvre was not suited to the pomp of a Médicis, and Catherine ordered a new palace to be built for her own special convenience in the Tuileries, or tile yards, where the mother of Francis I. had bought a country house, but where Francis I. would never reside, preferring to his Parisian residence the castles of Fontainebleau, Amboise, and Chambord. According to the plan of Philibert Delorme, the new Palace of the Tuileries was to be a true palace of the French kings, with a royal façade, the most beautiful gardens, and the most magnificent courtyards. Philibert Delorme never got beyond the façade, which, however, was enough to stamp him as an architect of the first order. Henri IV.—or rather Androuet Ducerceaux acting upon his orders—continued the work of Philibert Delorme. Ducerceaux made many changes, and among others constructed a dome where Philibert Delorme had meant only to build a cupola. Who, meanwhile, was to live at the Tuileries? It was a royal palace, but not the palace of the French kings. Valois did not live there, Catherine de Médicis gave magnificent entertainments at the Tuileries, but held her Court at the Louvre. Nor did Henri IV. reside at the Tuileries. His private apartments, decorated by the genius of Pierre Lescot, were at the Louvre, from which Paris could be better observed. Henri’s widow, Marie de Médicis, mourned for her generally excellent though not too faithful husband in the Luxembourg Palace. When Richelieu came to power and worked out the problem of the unity of France, he built the Palais Cardinal, but took no thought of the Tuileries. His eyes were fixed on the Louvre, where Louis XIII. was domiciled. Louis XIV. passed no more time at the Tuileries than any of his predecessors. His mother, Anne of Austria, established her regency at the Palais Cardinal, soon to {207} become the Palais Royal; and all idea of completing the Tuileries seemed to have been given up, when in 1660, under Louis XIV., then twenty-two years of age, the architects Levan and Dorbay were ordered to resume the work of Philibert Delorme and Ducerceaux—the work begun by Catherine, continued by Louis XIV.’s grandfather, Henri IV., and abandoned by his father, Louis XIII. The Palace of the Tuileries having at last been completed, it became the residence simply of Mlle. de Montpensier. From time to time Louis XIV. visited the
- 67. place, but only to make it the scene of some occasional entertainment. His favourite abode was always Versailles. While the Regent was at the Palais Royal, the youthful Louis XV. lived at the Tuileries. But as soon as he could walk alone, Louis le bien aimé, as he was afterwards to be called, hastened to Versailles; and the Tuileries Palace of strange destinies was now occupied by the French Opera Company. It became the Paris Opera House, the Académie Royale de Musique—to give the establishment its official title—whose theatre at the Palais Royal had been burnt down. In 1720 the Opera was replaced at the Tuileries by the Comédie Française. To Lulli succeeded Corneille and to Rameau Voltaire. One of the most interesting celebrations ever witnessed at the Tuileries was the crowning of Voltaire on the 30th of March, 1778, after a representation of his tragedy Irène. “Never,” wrote Grimm, the chronicler, in reference to this performance, “was a piece worse acted, more applauded, and less listened to. The entire audience was absorbed in the contemplation of Voltaire, the representative man of the eighteenth century; philosopher of the people, who could justly say, ‘J’ai fait plus dans mon temps que Luther et Calvin.’” Voltaire had but recently left Ferney to return to France, which he had not seen for twenty-seven years. Deputations from the Academy and from the Théâtre Français were sent to receive him, and on his arrival he was waited upon by men and women of the highest distinction, whether by birth or by talent. After the performance of Irène, he was carried home in triumph. “You are smothering me with roses,” cried the old poet, intoxicated with his own glory. The emotion, the fatigue, caused by the interesting ceremony, had indeed an injurious effect upon his health, and hastened his death, concerning which so many contradictory stories have been told. That he begged the curé of St. Sulpice to let him “die in peace” is beyond doubt; and that he died unreconciled to the Church, whose bigotry and persecution he had so persistently attacked, is sufficiently shown by the fact that, equally with Molière (though the great comedy writer had in his last moments demanded and received religious consolation), he was refused Christian burial. His nephew, the Abbé Mignot, had the corpse carried to his abbey of Scellières, where it remained until, under the Revolution, it was borne in triumph to the Panthéon. Eleven years after the crowning of Voltaire at the Tuileries, Louis XVI. arrived there from Versailles, where he had fraternised with the people, only to find that he was no longer a king. On the 19th of October, 1789, three
- 68. months after the taking of the Bastille, the National Assembly had waited in a body upon the king and queen, when the president, still loyal, said to Marie Antoinette: “The National Assembly, madame, would feel genuine satisfaction could it see for one moment in your arms the illustrious child whom the inhabitants of the capital will henceforth regard as their fellow- citizen, the offshoot of so many princes tenderly beloved by their people, the heir of Louis IX., of Henri IV., and of him whose virtues constitute the hope of France.” The queen replied, “Here is my son;” and Marie Antoinette, taking the young Louis in her arms, carried him into the room occupied by the Assembly. On the 26th of May, 1791, Barrère said to this same Assembly: “The first things to be reserved for the king are the Louvre and the Tuileries, monuments of grandeur and of indigence, whose plan, whose façades, are due to the genius of art, but whose completion has been neglected or rather forgotten by the wasteful carelessness of a few kings. Each generation expected to see this monument, worthy of Athens and of Rome, at last finished; but our kings, fearing the gaze of the people, went far from the capital to surround themselves with luxury, courtiers, and soldiers. It is characteristic of despotism to shut itself up in the midst of Asiatic luxury, as formerly divinities were placed in the depths of temples and of forests, in order to strike more surely the imagination of men. A great revolution was needed to bring back the people to liberty, and kings to the midst of their people. This revolution has been accomplished, and the King of the French will henceforth have his constant abode in the capital of the empire. This is our project. The Tuileries and the Louvre shall together form the {208} National Palace destined for the habitation of the king.” Thereupon the Assembly decreed: “The Louvre and the Tuileries joined together shall be the National Palace destined for the habitation of the king, and for the collection of all our monuments of science and art, and for the principal establishments of public instruction.”
- 69. THE TUILERIES IN THE EIGHTEENTH CENTURY. The position of the king at this time is well described by Arthur Young:— “After breakfast,” he writes in diary form, “walk in the gardens of the Tuileries, where there is the most extraordinary sight that either French or English eyes could ever behold at Paris. The king, walking with six Grenadiers of the milice bourgeoise, with an officer or two of his household, and a page. The doors of the gardens are kept shut in respect to him in order to exclude everybody but deputies or those who have admission tickets. When he entered the palace, the doors of the gardens were thrown open for all without distinction, though the queen was still walking with a lady of her court. She also was attended so closely by the gardes bourgeoises that she could not speak but in a low voice without being heard by them. A mob followed her, talking very loud, and paying no other apparent respect than that of taking off their hats whenever she passed, which was, indeed, more than I expected. Her Majesty does not appear to be in health; she seems to be much affected and shows it in her face; but the king is as plump as ease can render him. By his orders there is a little garden railed off for the Dauphin to amuse himself in and a small room is built in it to retire to in case of rain; here he was at work with his little hoe and rake, but not without a guard of two Grenadiers. He is a very pretty, good-natured looking boy, five or six years old, with an agreeable countenance; wherever he goes all hats are taken off to him, which I was glad to observe. All the family being thus kept
- 70. close prisoners (for such they are in effect) afford at first view a shocking spectacle, and is really so if the act were not absolutely necessary to effect the revolution. This I conceive to be impossible; but if it were necessary no one can blame the people for taking every measure possible to secure that liberty they had seized in the violence of a revolution. At such a moment nothing is to be condemned but what endangers the national {209} freedom. I must, however, freely own that I have my doubts whether this treatment of the royal family can be justly esteemed any security to liberty; or on the contrary, whether it was not a very dangerous step that exposes to hazard whatever had been gained.
- 71. I have spoken with several persons to-day and started objections to the present system, stronger even than they appear to me, in order to learn their sentiments, and it is evident they are at the present moment under an apprehension of an attempt toward a counter revolution. The danger of it very much, if not absolutely, results from the violence which has been used towards the royal family. The National Assembly was before that period answerable only for the permanent constitutional laws passed for the future;
- 72. since that moment it is equally answerable for the whole conduct of the government of the State, executive as well as legislative. This critical situation has made a constant spirit of exertion necessary amongst the Paris militia. The great object of M. La Fayette and the other military leaders is to improve their discipline and to bring them into such a form as to allow a rational dependence on them in case of their being wanted in the field; but such is the spirit of freedom that even in the military, there is so little subordination that a man is an officer to-day and in the ranks to-morrow; a mode of proceeding that makes it the more difficult to bring them to the point their leaders see necessary. Eight thousand men in Paris may be called the standing army, paid every day 15 fr. a man; in which number is {210} included the corps of the French Guards from Versailles that deserted to the people; they have also 800 horses at an expense each of 1,500 livres a year, and the officers have double the pay of those in the army.” If the people and the popular leaders were in constant fear of a counter revolution, the king on his side had had enough of royalty, and on the first opportunity fled from his subjects. The flight of the royal family, as is plainly shown by the correspondence of Marie Antoinette and by other authentic documents, had been concerted beforehand with the foreign Powers. This course was dictated by the most obvious considerations of personal safety. But all idea of an understanding with the “foreigner” was repudiated in the most solemn manner by the king. What the revolutionary Government resented was less the king’s desire to escape from a country where he had not only ceased to rule, but where his position was getting from day to day more precarious, than his apparent intention of making himself as soon as he had crossed the frontier the centre and support of a counter revolution. As the moment of departure approached, the king and queen renewed with increased energy protestations of their adhesion to the Constitution. At the same time the queen was writing to her brother Leopold, May 22nd, 1791: “We are to start for Montmédy. M. de Bouillé will see to the ammunition and troops which are to be collected at this place, but he earnestly desires that you will order a body of troops of from 8,000 to 10,000 to be ready at Luxembourg and at our orders (it being quite understood that they will not be wanted until we are in a position of safety) to enter France both to serve as example to our troops and if necessary to restrain them.”
- 73. On the 1st of June, after reiterating her demand for 8,000 or 10,000 troops at Luxembourg, close to the French frontier, she added: “The king as soon as he is safe and free will see with gratitude and joy the union of the Powers to assert the justice of his cause.” The plan, concerted with the Austrian ambassador at Paris, who had been the queen’s adviser, was first to place the royal family in safety beyond the French frontier, and then to act against France with an army of invasion aided within the country by a Royalist insurrection. It was at the same time understood that the Austrian Emperor and the German princes were not to give their aid gratuitously. They were to be recompensed by a “rectification” of the northern and eastern frontiers of France to their advantage. Troops were promised to Marie Antoinette by her brother Leopold, not only from Austria and various German States but also from Sardinia, Switzerland, and even Prussia. It was the popular belief at the time that Queen Marie Antoinette had determined to do some dreadful injury to Paris and other French cities; to blow them up, for instance, with gunpowder or by some secret means. At a village near Clermont in the Puy de Dôme, Arthur Young wished to see some famous springs; and the guide he had engaged being unable to render him useful assistance he took a woman to conduct him, when she was arrested by the garde bourgeoise for having without permission become the guide of a stranger. “She was conducted,” writes Young, “to a heap of stones they call the Château. They told me they had nothing to do with me; but as to the woman, she should be taught more prudence for the future. As the poor devil was in jeopardy on my account, I determined at once to accompany them for the chance of getting her cleared by attesting her innocence. We were followed by a mob of all the village with the woman’s children crying bitterly for fear their mother should be imprisoned. At the castle we waited some time, and we were then shown into another apartment, where the town committee was assembled; the accusation was heard, and it was wisely remarked by all that in such dangerous times as these, when all the world knew that so great and powerful a person as the queen was conspiring against France in the most alarming manner, for a woman to become the conductor of a stranger, and of a stranger who had been making so many suspicious inquiries as I had, was a high offence. It was immediately agreed that she ought to be imprisoned. I assured them she was perfectly innocent; for it was impossible that any
- 74. guilty motive should be her inducement. Finding me curious to see the springs, having viewed the lower ones, and wanting a guide for seeing those higher in the mountains, she offered herself; that she certainly had no other than the industrious view of getting a few sous for her poor family. They then turned their inquiries against myself—that, if I wanted to see springs only, what induced me to ask a multitude of questions concerning the price, value, and product of the land? What had such inquiries to do with springs and volcanoes? I told them that cultivating some land in England rendered such things interesting to me {211} personally; and lastly, that if they would send to Clermont they might know from several respectable persons the truth of all I asserted; and, therefore, I hoped, as it was the woman’s first indiscretion, for I could not call it offence, they would dismiss her. This was refused at first, and assented to at last, on my declaring that if they imprisoned her they should do the same by me and answer it as they could. They consented to let her go with a reprimand, and I started—not marvelling, for I have done with that—at their ignorance in imagining that the queen should conspire so dangerously against their rocks and mountains. I found my guide in the midst of the mob, who had been very busy in putting so many questions about me as I had done about their crops.” Such indeed was the general feeling against the king and queen, that, apart from other powerful motives, they had soon no alternative but to seek safety in flight. One of the principal agents in their escape was Count de Fersen, formerly colonel of the regiment of Royal Suédois. He was to drive the coach containing the king and queen. Marie Antoinette was to play the part of a governess, Mme. Rochet, in the service of an imaginary Russian lady, Baroness de Korff, impersonated by Mme. de Tourzel, actually governess to Marie Antoinette’s children. As for the king, disguised in livery, he was to pass as the Russian lady’s valet. The royal family was at this time confined more or less strictly to the Tuileries; and La Fayette, under whose command the troops on guard at the palace had been placed, had probably eyed with suspicion certain preparations made by the queen as if in view of a speedy departure.
- 75. LION IN THE TUILERIES GARDENS. (By Cain.) M. de Bouillé, who commanded at Metz, had orders to occupy the high road with detachments of troops as far as Châlons. During the night of the 20th of June, 1791, the royal family escaped from the Tuileries, reached La Villette, where Colonel de Fersen with a travelling carriage awaited them, and drove off towards Bondy, whence they were to make first for Châlons, and then for Montmédy, a frontier town. The next morning Paris woke up without a king. La Fayette, who had been wanting in vigilance, defended himself as best he could. An alarm gun was fired from the Pont Neuf to warn the citizens that the country was in the greatest danger, for it was quite understood that the passage of the frontier by the king and queen would be the signal for a foreign invasion. The National {212} Assembly met, and at once took into its hands the supreme direction of affairs. “This is our king!” said the Republicans; and Louis, by his flight, had in fact ceased to reign. Before leaving the Tuileries Louis XVI. had placed in the hands of La Porte, intendant of the civil list, a protest against the manner in which he had been treated, which was duly laid before the Assembly. Meanwhile, he had arrived at St. Ménéhould without accident, where he found himself protected by a detachment of dragoons which had arrived the night before. Here, however, his misfortunes began, for he was at once recognised by Drouet, a retired soldier now acting as postmaster. Called upon for horses, the young man could have no doubt but that the royal personages who required them were bound for the frontier, and he resolved to prevent their escape from France. With the dragoons in occupation of the
- 76. village he could not refuse to supply horses; and the carriage which bore Louis and his fortunes, now approaching the end of its critical journey, went off in an easterly direction. Scarcely had the post chaise departed when Drouet, aided by a friend named Guillaume, also a retired soldier, called out by beat of drum the local national guard, and ordered it to prevent the dragoons from leaving the village. He then, together with Guillaume, galloped after the royal carriage, followed by a sub-officer of dragoons named Lagache, who, escaping from St. Ménéhould, had resolved to catch them up, and, if possible, kill them. Riding along, Drouet learned that the carriage had taken the road to Varennes, a town which has twice played an important part in the history of France, for it was here, seventy-nine years later, that the King of Prussia established his head-quarters on the eve of the battle of Sedan. {213} THE CHESTNUTS OF THE TUILERIES. By crossing a wood Drouet and Guillaume succeeded in getting to Varennes a trifle sooner than the royal carriage. Passing, at no great pace, the lumbering vehicle just as it was approaching the town, they at once made for the bridge on the other side of Varennes, which, as old soldiers, they saw the necessity of blocking, for beyond it, on the other side of the river Aire, they had discovered the presence of a detachment of cavalry under the command of a German officer, who, losing his head, took to flight. The energetic Drouet had already waked up the town, and, in particular, the principal
- 77. officials, such as the Mayor, the {214} Procureur of the Commune, c. The population answered to Drouet’s call, and soon a small body of armed men was on foot. LOUIS XVI. STOPPED AT VARENNES BY DROUET. The fugitives were bound for the Hôtel du Grand Monarque. At this hotel a tradition is preserved which was communicated to the present writer by the proprietress, Mme. Gauthier, just before the battle of Sedan. Dinner was prepared there for Louis XVI. eight days running; from which it would appear that he was trying to escape from the Tuileries for eight days before he at last succeeded in getting away unobserved. The eighth, like all the preceding dinners cooked for the unfortunate king at the Hôtel du Grand Monarque, was destined to remain uneaten. It was now late at night, and
- 78. when the royal carriage entered the town, it was surrounded in the darkness by a number of armed men, who asked for passports, and showed by their attitude that they had no intention of allowing the occupants of the vehicle to proceed any further. Emissaries from Varennes had been despatched in all haste to the surrounding villages and nearest towns to call out the national guard. The son of M. de Bouillé had meantime quitted the cavalry outside Varennes, and ridden towards Metz to inform the governor, his father, of the arrival of the fugitives. But when the commandant arrived outside Varennes with an entire regiment of cavalry, the town was occupied by 10,000 infantry, and all the approaches guarded in such a manner that it was impossible for de Bouillé’s regiment to act. The Procureur, to whose house the royal family had been taken, informed the king in the early morning that he was recognised. A crowd, which had gathered before the house, called for him by name, and when Louis showed himself at the window he understood from the attitude of the mob that though he was saluted here and there with cries of “Vive le Roi!” there was an end to his project of reaching the frontier. At six o’clock couriers arrived from Paris with a decree from the Assembly ordering the king’s arrest; and at eight o’clock on the morning of the 22nd of June, 1791, the royal family started under escort for the capital. They were surrounded at the moment of departure by an immense mob, a portion of which followed them for some distance along the road. At Epernay the commissaries appointed by the Assembly, MM. Pétion and Barnave, were waiting to take the direction of the cortege. On being questioned the king declared that he had never intended to leave the kingdom, and that his object in retiring to Montmédy had been to study the new Constitution at his ease, so that, with a clear conscience, he might be able to accept it. Barnave and Pétion got into the royal carriage as if to prevent all possibility of escape. Louis was treated with all the respect due to a royal captive, but his position was that of a prisoner. Reaching Paris three days after his departure from Varennes, he was received by the people with the greatest coldness. On the walls of the streets through which he passed, these words had been inscribed: “Whoever applauds Louis XVI. will be beaten; whoever insults him will be hanged.” To avoid the popular thoroughfares, the Tuileries was approached by way of the Champs Élysées, and once more Louis took up his abode in the ancient palace of the French kings.
- 79. Differences between Louis XVI. and the Assembly, which, from “Constituent” had become “Legislative,” now suddenly occurred; and at the beginning of 1792 the Jacobin Rhul complained from the tribune that the king had treated with disrespect certain commissaries of the Assembly who had waited upon him. On the 25th of July of the same year the king was accused in the Chamber of collecting arms at the Tuileries. National guards, it was said, went in armed and came out unarmed; and it was declared to be unsafe for the National Assembly to have an arsenal of this kind in its immediate neighbourhood. Accordingly, the Assembly decreed that the terrace of the Tuileries gardens must be regarded as its property, and be placed beneath the care of the Assembly’s own police. The king objected, naturally enough, to the gardens of his palace being thus interfered with. “The nation,” said one of the deputies, “lodges the king at the Palace of the Tuileries, but I read nowhere that it has given him the exclusive enjoyment of the gardens.” Some days afterwards the same deputy, Kersaint by name, said from the tribune: “The Assembly having thrown open one of the terraces of the Tuileries gardens, the king, who does not think fit to render the rest of the gardens accessible to the public, has lined the terrace with a hedge of grenadiers.” Chabot called the garden of the Tuileries “a second Coblentz,” in reference to the German fortified town where the allied sovereigns, who were plotting against the Revolution, had their head-quarters. On the 19th of August a journeyman painter named Bougneux sent word to {215} the Assembly that there had recently been constructed in the Palace of the Tuileries several masked cupboards. Three months afterwards Roland brought to the Convention the papers of the famous iron cupboard. “They were concealed,” he said, “in such a place, in such a manner, that unless the only person in Paris who knew the secret had given information it would have been impossible to discover them. They were behind a panel,” he continued, “let into the wall and closed in by an iron door.” The members of the Mountain, as the extreme party occupying the highest seats in the legislative chamber were called, accused Roland of having opened the metallic cupboard in order to make away with the papers of a compromising character for his friends the Girondists. In revolutionary times a good action may be as compromising as a bad one. Brissot proposed about this time that the meetings of the Convention should be held at the Tuileries. Vergniaud had preferred the Madeleine. “Not,” he said, “in either case, that liberty has
- 80. need of luxury. Sparta will live as long as Athens in the memory of nations; the tennis court as long as the palaces of Versailles and of the Tuileries. The external architecture of the Madeleine is most imposing. It may be looked upon as a monument worthy of liberty, and of the French nation.” It need scarcely be explained that at the jeu de paume, or tennis court, the first revolutionary meetings were held. “At the Tuileries,” said Brussonnet, “there is a finer hall; and the greater the questions which the National Assembly will have to treat the greater must be the number of hearers and spectators.” It was at last decreed that the Minister of the Interior should order the preparation at the Tuileries of a suitable hall for the debates of the National Convention; and with that object a sum of 300,000 francs was voted. On the 4th of September, 1793, Chaumette, in the name of the Paris commune, appeared at the bar of the Convention, then presided over by Robespierre, and spoke as follows: “We demand that all the public gardens be cultivated in a useful manner. We beg you to look for a moment at the immense garden of the Tuileries. The eyes of republicans will rest with more pleasure on this former domain of the crown when it is turned to some good account. Would it not be better to grow plants in view of the hospitals, than to let the grounds be filled with statues, fleurs de lis, and other objects which serve no purpose but to minister to the luxury and the pride of kings?” Dussaulx added with a smile: “I demand that the Champs Élysées be given up at the same time as the gardens of the Tuileries to useful cultivation.” It was at the Tuileries that the Committee of Public Safety held its meetings: that irresponsible body which struck so many and such sanguinary blows at the accomplices, real or imaginary, of invasion from abroad, and of insurrection at home. In the Tuileries gardens took place the festival of the Supreme Being, when proclamation was solemnly made, under the authority of Robespierre, that the French people believed in God and the immortality of the soul. “People of France,” cried Robespierre, between two executions, “let us to-day give ourselves up to the transports of pure unmingled joy. To- morrow we must return to our progress against tyranny and crime.” To Robespierre’s passionate declamation succeeded solemn music, composed by Méhul. Soon afterwards Tallien, inspired to an act of daring by the news that the woman he loved and afterwards married had been condemned to death, denounced Robespierre; and it was at the Tuileries that the Reign of Terror, like so many other reigns, came to an end.
- 81. On the 1st of February, 1800, Bonaparte took possession of the Tuileries, with his wife Joséphine. In 1814 he quitted the ancient palace with Marie Louise. The Tuileries was now on the point of being occupied by foreigners. “When I returned to Paris,” writes Mme. de Staël, “Germans, Russians, Cossacks, Baskirs, were to be seen on all sides. Was I in Germany or in Russia? Had Paris been destroyed and something like it raised up with a new population? I was all confusion. In spite of the pain I felt I was grateful to the foreigners for having shaken off our yoke. But to see them in possession of Paris! to see them occupying the Tuileries!” Louis XVIII. and Charles X. both reigned at the Tuileries. But in July, 1830, the Revolution once more took possession of the palace; and in 1848, after the flight of Louis Philippe, the mob again ruled for a time in the home of the French kings. In 1848 the Provisional Government converted the Tuileries into an asylum for civilians. But the conversion was made only on paper, and in 1852 the Tuileries became for the second time an imperial palace—the palace of Napoleon III. The fate of the historical structure was, as everyone knows, to be burnt by the Communards. It was on the 24th of May, 1871, when the Versailles {216} troops were already in the Champs Élysées, that the central dome of the palace, the wings, the whole building in short, was seen to be in flames. The new portions of the palace alone refused to burn. Then, in their rage, the incendiaries had recourse to gunpowder, and during the night a formidable explosion was heard. The troops of the Commune, commanded by the well-known General Bergeret, had retired some hours before. Bergeret, however, was not responsible for the incendiarism; and the person afterwards tried for it and condemned to hard labour for life (in commutation of the death punishment to which he was first sentenced) was a certain Benoit, formerly a private in the line, then, during the siege, a lieutenant in the National Guard, and finally colonel under the Commune.
- 82. THE ROYAL FAMILY AT VARENNES. The gardens of the Tuileries are now more than ever open to the reproach brought against them by the men of the Revolution, who objected to statues adorning its terraces and walls, and wished its works of art to be replaced by lettuces and cabbages. All the greatest sculptors of France are represented in the Tuileries gardens, which also contain many admirable reproductions of ancient statues and groups. There is one interesting walk in the Tuileries gardens which is the favourite resort of children. Here it was, in the so-called petite Provence, that the children’s stamp exchange was established, against which the authorities found it necessary to take severe steps. The young people have since contented themselves with balls, balloons, and other innocent amusements. There is a Théâtre Guignol, moreover, a sort of Punch and Judy, in the middle of the old gardens; and from the {217} beginning of April to the middle of October a military band plays every day. It is impossible to leave the Tuileries gardens without mentioning its famous chestnut tree—the chestnut tree, as it is called, “of the 20th of March,” because in 1814 it blossomed on that very day as if to celebrate Napoleon’s return from Elba. But the old chestnut tree had a reputation of its own long before the imperial
- 83. era. More than a hundred years ago the painter Vien, at that time pupil of the French School, was accused of having assassinated a rival who had competed with him for a prize. He was about to be arrested when he proved that at the very hour when the crime must have been committed he was tranquilly seated beneath the future “chestnut tree of the 20th of March,” which was distinguished just then from all the other trees in the garden by being alone in flower. This picturesque alibi saved his life. MONUMENT TO GAMBETTA, PLACE DU CARROUSEL. Outside the remains of the Tuileries was erected, on the Place du Carrousel, in 1888, a monument to Gambetta. The design as a whole has
- 84. been unfavourably criticised, but the figure of the orator himself, represented in the act of declamation, is bold and striking, and full of character. {218}
- 85. B CHAPTER XX. THE CHAMPS ÉLYSÉES AND THE BOIS DE BOULOGNE. The Champs Élysées—The Élysée Palace—Longchamp—The Bois de Boulogne—The Château de Madrid—The Château de la Muette—The Place de l’Étoile. EFORE entering the Champs Élysées, the greatest pleasure thoroughfare in Paris, next to, if not before, the line of boulevards, a brief examination of the frontiers, as approached from the Place de la Concorde, may be advisable. This region of the capital was for a long time one of those marshes by which ancient Paris, the Lutetia of the Romans, was enclosed like a fortress. Then it became cultivable land and passed into the hands of market gardeners, who grew their vegetables in fields by no means “elysian,” until the latter part of the reign of Louis XV. The ancient marsh was bounded on one side by the Seine, on the other by the Faubourg St. Honoré, which in the eighteenth century was already a favourite locality for mansions of the nobility. The market gardens, more fertile, perhaps, by reason of their marshy origin, were traversed by the Chemin du Roule—so named from the slope called rotulus, in the days of Lutetia, of which the culminating point is now marked by the Triumphal Arch. At the entrance to the Champs Élysées stands the celebrated marble group known as the Horses of Marly; and close to the entrance is the garden of the Élysée Palace (Élysée Bourbon, to call it by its historical name), whose principal gates open into the Rue du Faubourg St. Honoré. Built in 1718 by the architect Mollet on a portion of the St. Honoré marshes which had been given by the Regent to Henri de la Tour d’Auvergne, Count of Evreux, the Élysée Palace passed in 1745 from the count’s heirs to Madame de Pompadour. Her brother, the Marquis de Marigny, inherited it from her, and, holding the appointment of Inspector and Director of Royal Buildings, he embellished the palace and made great improvements in that portion of the neighbourhood known to-day as the Champs Élysées. It was now only that the mansion, called successively Hôtel d’Evreux, Hôtel de Pompadour, and Hôtel de Marigny, received the name of Élysée.
- 86. Towards the period of the Revolution, in 1786, the Élysée Palace was purchased by the king, and, according to the terms of a royal decree, was to be reserved for the use of princes and princesses visiting the French capital as well as ambassadors charged with special missions. Almost immediately afterwards, however, the structure was bought by the Duchess of Bourbon, when Élysée Bourbon became its recognised name. This very appellation was enough to condemn it in the days of the Revolution; and the Duchess of Bourbon having migrated, her property was seized and confiscated. Sold by auction, it was acquired by Mlle. Hovyn, who seven years later ceded it to Murat; and Murat, on leaving Paris to assume the crown of Naples, presented it to the emperor. Napoleon accepted the gift and took a fancy to his new edifice. He often resided there; and after the defeat of Waterloo it was at the Élysée that he signed his abdication in favour of his son. In 1814 and 1815 the Élysée was temporarily occupied by Alexander I. of Russia. At the Restoration, the Duchess of Bourbon, returning to France, claimed her property. Her rights were recognised, but she was prevailed upon to accept, in lieu of the Élysée, the Hôtel de Monaco in the Rue de Varennes, which she left by will to the Princess Adelaide of Orleans, sister of Louis Philippe. Under the Restoration, it was at the Élysée, now called once more Élysée Bourbon, that the Duke and Duchess of Berry resided until 1820, when, after the assassination of the duke, the duchess felt unable to live there any longer. The duke and duchess were the last permanent tenants of the Élysée, which under the reign of Louis Philippe was utilised, in accordance with the intentions of Louis XVI., as a resting-place for royal guests, or guests of the first importance. In its new character it received Mahomet Ali Pasha of Egypt, and Queen Christina of Spain. After the 10th of December, 1848, Prince Louis Napoleon, elected President of the Republic, had the Élysée assigned to him as his official place of residence. It was here that the coup d’état of the 2nd of December, 1851, was planned and plotted by the Prince-President, {219} and the Count de Morny, his minister, confidant, and guide, General St. Arnaud, and other accomplices. On proclaiming himself Emperor, Napoleon III. gave up possession of the Élysée, and removed to the more regal, more imperial palace of the Tuileries; the Élysée, being now once more set apart for foreign potentates and other grandees visiting Paris. Under the Second Empire
- 87. Queen Victoria, the Sultan Abdul Aziz, and the Emperor Alexander II. of Russia, were successively received there. Since the establishment of the Third Republic the Élysée has been made the official residence of the President; and it has been inhabited, one after the other, by M. Thiers, Marshal MacMahon, M. Grévy, and M. Carnot. It has been said that the Élysée Palace stands between the Rue du Faubourg St. Honoré and the Champs Élysées, with its principal entrance in the street. Between these two thoroughfares stood the ancient Village du Roule, which possessed, as far back as the thirteenth century, an asylum for lepers with a chapel attached to it. This chapel was in 1699 elevated to the rank of parish church, under the invocation of St. Philip. Being now too small it was pulled down; and in place of it was built the present church of St. Philippe du Roule, which underwent a partial transformation in 1845 and 1846. The principal avenue of the Champs Élysées was planted with trees in 1723; but it was not until the reign of Louis XVI. that the Champs Élysées, or rather that portion of the avenue known as Longchamp, became a haunt of fashion. The so-called promenade of Longchamp was, towards the end of the eighteenth century, frequented by the most aristocratic society. Gradually after the Revolution it got to be a more miscellaneous resort, to become ultimately, in modern times, a sort of show ground for fashionable milliners and dressmakers, hatters and tailors. The Abbey of Longchamp, whence the promenade derived its name, was founded as a convent in the thirteenth century by Isabelle of France, sister of Louis IX., and pulled down at the time of the Revolution. It was situated close to the Bois de Boulogne, near the village of that name. “I wish to ensure my salvation,” wrote the Princess Isabelle to Hémeric, Chancellor of the university, “by some pious foundation. King Louis IX., my brother, grants me 30,000 Paris livres, and the question is, shall I found a convent or a hospital?” The Chancellor’s advice was to establish an asylum for the nuns of the order of St. Clara. In 1260 Isabelle built the church, the dormitories, and the cluster of the Humility of Our Lady; and according to Agnes d’Harcourt, who has written her life, the whole of the 30,000 livres was consumed. The year afterwards, on the 23rd of June, the nuns of the rule of St. Francis took possession of the abbey in presence of Louis IX. and all the Court. The king gave considerable
- 88. property to the nuns, whom he often visited, and, by his will, dated February, 1269, this sovereign, on the point of undertaking his last expedition to Palestine, left a legacy to the Abbey of Our Lady. Isabelle in this very year ended her days within its walls. The royal origin and associations of the house which the princess had founded ensured for it the patronage of successive French sovereigns— Marguerite and Jeanne de Brabant, Blanche de France, Jeanne de Navarre, and twelve other princesses, taking the veil there; and it is recorded that Philippe le Long died in it with his daughter Blanche by his side on the 2nd of December, 1321, of complicated dysentery and quartan fever. When he was approaching his end the abbé and monks of St. Denis came in procession to his aid, bringing with them a piece of the True Cross, a nail that had been used at the Crucifixion, and one of the arms of St. Simon. The exhibition and application of these pious relics gained for the king enough time to make his will, after which he expired. Longchamp had no fewer than forty nuns in residence. Its proximity to Paris, its illustrious origin, its not less illustrious visitors, its aristocratic inhabitants, its vicissitudes during the sanguinary civil wars of the fifteenth and sixteenth centuries, its decline, and, ultimately, its ruin, invested it with extraordinary interest. As regards the history of the abbey, it must be mentioned that, as with all other convents, its discipline gradually became relaxed until at last purity gave way to licence. Henri IV. took from Longchamp one of his mistresses, Catherine de Verdun, a young nun of twenty-two, to whom he gave the priory of St. Louis de Vernon, and whose brother, Nicholas de Verdun, became first President of the Parliament of Paris. “It is certain,” wrote St. Vincent de Paul, on the 25th of October, 1652, to Cardinal Mazarin, “that for the last 200 years this convent has been gradually getting demoralised until now there is less discipline there than depravity. Its reception rooms are open to anyone who comes, {220} even to young men without relations at the convent. The order of friars (Cordeliers) under whose direction it is placed, do nothing to stop the evil. The nuns wear immodest garments and carry gold watches. When, war compelled them to take refuge in the town the majority of them gave themselves up to all kinds of scandals, going alone and in secret to the men they desired to visit.” It is evident from this letter that there were intimate relations between the Abbey of Longchamp and Paris. It had been the custom, moreover, since the
- 89. fifteenth century, to go to Longchamp to hear the friars of the order of Cordeliers preach during Lent. “In 1420,” says the journal of Charles VII., “Brother Richard, a Cordelier, lately returned from Jerusalem, preached such a fine sermon that the people from Paris who had been to hear it made more than one hundred fires on their return—the men burning tables, cards, billiard-tables, billiard-balls, and bowls; while the women sacrificed head-dresses, and all kinds of body ornaments, with pieces of leather and pieces of whalebone, their horns and their tails.” A great many miracles were said to take place through invocations addressed to the Princess Isabelle, whom Pope Leo X., by a bull dated January 3, 1521, had canonised; while he, at the same time, granted to the nuns of Longchamp the privilege of celebrating annually, in her honour, a solemn service on the last day of August. From the early days of the reign of Louis XV. date those regular pilgrimages to Longchamp during Holy Week, which were soon to degenerate into mundane promenades.
- 90. THE HORSES OF MARLY, CHAMPS ÉLYSÉES. At one time the singing of the nuns had been found attractive. In 1729 a vocalist from the Opera, Mlle. Lemaure, sang with the choir, and “all Paris” went to hear her. The nuns profiting by her lessons, and studying her style, sang the “Tenebræ” during Holy Week with so much success that in order to make the choir perfect the abbess applied to the Opera for some additional voices. The abbey was now more than ever besieged. People crowded round the walls, filled the churchyard, and, according {221} to one writer, stood on the tombstones. If the chorus-singers from the Opera were not converted to piety by the nuns, the nuns underwent the influence of the professional vocalists. At last, one Wednesday in Holy Week, a brilliant gathering of
- 91. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com