Eigenfaces

EIGENFACES FOR RECOGNITON

Paper: EigenFaces For Recognition, 1991

Authors: Matthew Turk and Alex Pentland

Presenter: Semih Korkmaz

1/24

EigenFaces For Recognition, 91



Left: Prof. Dr. Matthew Turk, currently working at UC Santa Barbara
University(http://transliteracies.english.ucsb.edu) Right :Prof. Dr. Alex
Pentland, Currently working at MIT. (http://ticsp.cs.tut.fi)
2/24



Contents

• Overview of the method
• Principal Component Analysis

• Recognition Process
• Acquiring Images
• Calculating EigenFaces
• Training the system

• Additional Capabilities

• Conclusion and Recent Work

3/24


Overview
• Acquire training images.
• Calculate Eigenfaces.

• Project them to face space.
• Project test image to face space.

• Calculate the Euclidean distance and make
a decision.
4/24



Principal Component Analysis
Find the dimensions of data with highest variance

http://web.media.mit.edu/


5/24


Principal Component Analysis
Finding patterns in many dimensions is hard.
Mapping to a simpler domain is desirable.
𝑛 → 𝑘 | 𝑘≪ 𝑛
𝑛, 𝑘 number dimensions

Invented in 1901, by Karl Pearson.

6/24



Acquire Training Images
Get 𝑀 training samples with variances

…
𝐼1

𝐼2

𝐼3

𝐼4

…

𝐼 𝑀−1

𝐼𝑀

(Olivetti - Att – ORL dataset, ‘94)

Images are in same size and equivalently framed.
7/24



Calculate EigenFaces
• Convert all the images in vector form.
25

55

8

200

180

70

𝐼𝑖 =

40
65

Γ𝑖 ′ =

18

25

40

55

8

200 180 70

65

18

𝑁2

𝑁 × 𝑁

• Calculate the mean . (Average Face)
1
Ψ=
𝑀

𝑀

Γ𝑛
𝑛=1

8/24


• Normalize vectors.
Φ𝑖 = Γ 𝑖 − Ψ

• Form the covariance matrix
𝐴 = [Φ1 , Φ2 , . . , Φ 𝑚 ]
1
𝐶=
𝑀

𝑀
𝑇
Φ 𝑛 Φ 𝑛 = 𝐴𝐴 𝑇
𝑛=1


9/24


• We calculate the Eigen vectors of Covariance Matrix
𝐶 = 𝐴𝐴 𝑇 → 𝑁 2 × 𝑀 . 𝑀 × 𝑁 2 → 𝑵 𝟐 × 𝑵 𝟐

• Do we need so many eigenvectors anyway ?
No, we don’t ! Calculate eigenvectors of the
Covariance matrix with reduced dimensionality.
10/24



𝐶 = 𝐴 𝑇 𝐴 → 𝑀 × 𝑁2. 𝑁2 × 𝑀 → 𝑴 × 𝑴
𝑣 𝑖 is an eigenvector of 𝐴 𝑇 𝐴
𝜇 𝑖 is an eigenvector of 𝐴 𝐴 𝑇 (Eigen Face)
𝜇 𝑖 = 𝐴𝑣 𝑖
(𝐴 𝑇 𝐴)𝑣 𝑖 = 𝜆 𝑖 𝑣 𝑖

𝐴𝐴 𝑇 𝐴𝑣 𝑖 = 𝜆 𝑖 (𝐴𝑣 𝑖 )
Calculate 𝑘 eigenvectors and associate remaining to 0.
11/24



Training the system
Ψ
=

𝜇1 * 𝜔1

+

6 eigenfaces
case

+

𝜇2 ∗ 𝜔2 +

𝜇3 ∗ 𝜔3 +

𝜇4 ∗ 𝜔4 + 𝜇5 ∗ 𝜔5 +

𝜇6 ∗ 𝜔6
12/24



Training the system
• Images projected to face space.
𝜔 𝑘 = 𝜇 𝑘𝑇 (Γ − Ψ)

• Images projected to face space.
Ω𝑖 =

𝜔1
𝜔2
𝜔3
…
𝜔𝑘

𝑀′

Φ𝑓 =

𝜔𝑖 𝜇𝑖
𝑖=1

13/23



Training the system
Testing a face has two cases:

• Find the nearest face with designated threshold 𝜃 𝜖
𝜖2 = ( Ω − Ω 𝑘)
𝑘

2

compare with 𝜃 𝜖

• Normalize and find out if it is a face according to𝜃 𝜖
2

𝜖 = ( Φ − Φ 𝑓)

2

compare with 𝜃 𝜖

14/23



Additional Capabilities
Detection and Tracking

• Check around every pixel for an image
• Try to classify faces using spatiotemporal filtering
for a video
• Both methods can be combined
15/23



Relation to Neural Networks

• Model the system as Neural Network.
Φ

Ω

Φ𝑓

16/23



Increasing Robustness

• Multiply around the face with Gaussian for
attenuating the effects of background.
• Try different scales of eigenfaces, estimate head
pose.

• Up to 45 𝜊 turned faces with profile might be
interpolated.
17/23



Summary
1. Acquire a set images with variations
2. Calculate eigenfaces and choose M’ of them
associated with highest eigenvalues.

3. By projecting each indivual’s images to face space,
train the system.
4. Given a test image; project it to face space and
make decision according to threshold.
18/23



Results
Percentage results for Recognition from AT&T dataset, equal number
of training and test images.
96
90

94

85

92

80

EigenFaces

90

75

Fisher Faces

88

LBP

86
70
10 Faces 50 Faces

100
Faces


84
r:1 n:8 r:2 n:8 r:1 n:8
nx:8 nx:8 nx:4
19/23
ny:8 ny:8 ny:4

Results
Speed of Eigenfaces, 200 images for training and testing.
Eigenfaces

Training+Test

Test

10

0.52 seconds

0.02 seconds

50

0.7

0.11

100

0.92

0.25

20/23



Results

(Caltech Face Dataset,’99)

Selection of 150 images from Caltech Faces(Converted to
Grayscale);
45 Training, 105 test and 10 eigenfaces selected. Eigenfaces
used directly and..
Only 7 (!) are correctly classified.

21/23


References
[1]Matthew Turk and Alex Pentland. 1991. Eigenfaces for recognition. J.
Cognitive Neuroscience 3, 1 (January 1991), 71-86.
DOI=10.1162/jocn.1991.3.1.71 http://dx.doi.org/10.1162/jocn.1991.3.1.71
[2]L. Sirovich and M. Kirby, Low-dimensional Procedure for the
Characterization of Human Faces, Journal of the Optical Society of
America A, 4:519--524, 1987
[3]Belhumeur, P.N.; Hespanha, J.P.; Kriegman, D., "Eigenfaces vs.
Fisherfaces: recognition using class specific linear projection," Pattern
Analysis and Machine Intelligence, IEEE Transactions on , vol.19, no.7,
pp.711,720, Jul 1997 doi: 10.1109/34.598228
22/23



References
[4] Anil K. Jain and Stan Z. Li. 2005. Handbook of Face Recognition. SpringerVerlag New York, Inc., Secaucus, NJ, USA.

23/23




Thank you for listening
Questions ?



Eigenfaces

More Related Content

What's hot (20)

Similar to Eigenfaces (20)

Recently uploaded (20)

Eigenfaces