Matrix mirrors project

Matrix mirrors – a very short presentation

Renato Roque – FEUP, University of Porto

This short text presents very briefly Matrix Mirrors project and some of its results. Matrix Mirrors was driven by a
Multimedia Master Thesis in FEUP (2007/2009) and it led as well to a photographic/artistic project with the same
name. (please see attached documentation on artistic project).

We are looking for ideas in order to continue our work. One main vector for this continuation would be to guarantee
that it would cover not only technical/ scientific/ image processing issues but it could be as well the basis for the
continuation of the photographic/ artistic project, which was an important result of our previous work.

Matrix mirrors – project background

Since the 1950’s that a number of psychologists and neurologists believe that Humans have
developed specialized mechanisms to learn, memorize and recognize faces. They support that this
sophisticated learning and recognition process was an essential achievement for the specie’s
survival.

Figure 1 – Areas in the brain involved in face recognition process (red striped)

Concepts and theories from the communication area, such as entropy, sparse coding, redundancy
and signal/noise relation were adopted to quantify and try to explain a lot of experimental data,
related with visual perception, in particular the perception of human faces. In particular
redundancy concept has played a central role in this vision. Horace Barlow was the first to draw
the importance of such concept for the economy in representation and for the speed in recognition.
He believed that a mechanism to identify regularities or redundancies was crucial to optimize this
process. Most information in a new face will correspond to information in other faces. It is
redundancy which allows unsupervised learning process. On the other hand, to be able to recognize
something new, the system must allow comparing what is being observed with what is usually
seen.

Related to this vision there has been in the computational areas, since the 1990’s, a lot of
investigation to use statistical tools for image-vectors, to create automatic recognition applications.
There has been a lot of work to develop valid models and efficient algorithms for the processing of
natural images and in particular human faces.

There are already today, for example, a few commercial products making use of this work, for
example to identify a person from a photo or from a video.

1

Matrix mirrors – a photographic project

Photography has been associated since its invention to something involving mystery and magic. Its
capability of freezing time and space has still today, in spite the mass vulgarization of photos, an
aura of witchcraft.

Photography has been clearly associated since its invention in the XIXth century with identity.
Photographs have been and are still used in all identity documents, like passports and all kinds of
cards. Looking at human portraits trough statistic techniques will allow a reflection around human
identity and what it means. Concepts such as entropy will allow quantifying answers to questions
such as: What is common and what is different in each human face? What new information do we
get when we know a new face?

The knowledge that it might be possible to use mathematical/image processing tools to get, from a
portrait database, a set of abstract components, which would allow us to reconstruct any portrait,
within or without the database used, just adding those components in the right percentage, was
something completely magic to add to the magic of photography. Still today, this capability is a
wonder to us. Thinking that we might have in our brains an equivalent process is still a bigger
wonder.

Matrix mirrors – some results

All computational work which we referred makes use of sophisticated statistical/ data/ image
processing techniques like PCA (Principal Component Analysis), ICA (Independent Component
Analysis) or NMF (Non negative Matrix Factorization) which allow, using different criteria,
decomposing a portrait into a set of components. We could say that these components correspond
to portrait’s global abstract features. If we turn them into images they can be seen as ghostly faces.
An example can be seen in Figure 2, which shows some of these components, which we calculated
from our portrait Data Base (DB) using PCA.

Figure 2 – PCA components - Eigen faces

In our work we used such techniques but from a photography point of view. We wanted to evaluate
and compare the capability of such techniques, not to produce a correct machine identification of a
portrait, comparing coefficients, not to improve the recognition rate of existing algorithms, but to
reconstruct recognizable human portraits, using generic components.

We photographed 439 persons from the Oporto University and built a 400 facial portraits 200x200
pixels database. This resolution was imposed by our computer memory, taking into account the
huge size of the matrixes to deal with. The other 39 portraits were used as a test set to simulate the
behaviour of the system towards new portraits. We calculated statistical components for our
database, using PCA, ICA, NMF and PCA+ICA (a new hybrid statistical system).

2

These components can be added to reconstruct, not only the portraits which were used to calculate
them, but other new portraits as well, using simple formula like:

I = I DC + ∑ Ak I k (1)
k
First we proved that, with the right coefficients, all 400 portraits from the database can
reconstructed with perfection and we need even only a few components to perform identification:
with an hybrid PCA+ICA system (the most performing) we only need less than 20 components to
succeed in identification.

We concluded that even new portraits, like the one in Figure 3, which had not been used to
calculate the components, can be reconstructed, although the reconstruction is not perfect this time.
In spite of the final error, these reconstructed portraits can be recognized once more making use of
only about 30 components.

Using a questionnaire we demonstrated that with only a few components (about 20 for portraits in
the database and 30 for new portraits) we can reconstruct all portraits, with an error low enough to
allow recognition. We observed that an error below 4.5% allows nearly 100% recognition rate.
Error in reconstruction was measured using Euclidean distance concept. These values were
obtained with PCA+ICA hybrid system and with a normalized database, where portraits were
normalized to guarantee that both eyes from all portraits are coincident. Other statistical techniques
even with a non normalized database show similar results, they only require a slight bigger number
of components.

Figure 3 – The reconstruction of a new portrait, which was not in database step by step

Another apparently very significant result was that we observed that a portrait from a specific
group, for example a woman, can be reconstructed without any visible difference, using
components calculated from a DB of portraits from a different group, for example men. Women
can be reconstructed as well as men from a DB of men's portraits. As well an African face can be
reconstructed as well as an European face from an European DB.

Matrix mirrors – some statistical results

Using our portrait database a few statistical portraits could be calculated and some very interesting
results were obtained. We calculated namely average, standard deviation (SD), skew and kurtosis
portraits for the whole database and for attribute related groups. Figure 4 shows the DB average
portrait.

3

Figure 4 – Average portrait for the 400 portraits in the database

Figure 5 shows average and SD portraits for different groups, which we considered in our work.

Figure 5 – Average and SD portraits for different groups
From left to right: whole DB, men, women, 118 men, men aged over 50, men aged less than 30

We observe clearly that on one hand the whole DB statistic portraits appear to be very similar to
men's portraits and that is not due to the fact that we have more men than women in our DB,
because we obtain the same result with 118 men, the number of women in our DB, as we can see.
On the other hand, according to what one might expect, we can observe that each group appears to
lead to specific statistical portraits, where one can observe some group's characteristics.

But we decided as well to analyse the evolution of statistical portraits with DB dimension, mixing
all kinds of portrits: men, women, old and young people. Figure 6 shows the results for average and
SD portraits.

4

Figure 6 – Average and SD portraits for different DB dimensions
From left to right: 6, 25, 50, 100, 200 and 400 portraits

It appears to be very relevant to observe that we need only about 50 persons to get a result which is
very, very similar with the statistical portraits for the whole DB, which could lead us to think that
this might be the humanity's average and SD portraits!

Although each group shows to have its statistical specificities, when we mix different portraits, we
come very fast to average and SD generic portraits.

Matrix mirrors – some ideas to continue previous work

We would like to enlarge our portrait DB, in order to make it equally representative of men and women and
of different ethnical origins, allowing this way to validate some of the results which we intuited in our work.

Already after completion of the Master Thesis we have been trying some interesting tools to perform
automatic image classification - cluster analysis of portraits - based upon portraits dissimilarity.

Matrix mirrors – Main references

[1] Aapo Hyvärinen. 1999. Fast and Robust Fixed-Point Algo-rithms for Independent Component Analysis. . IEEE Trans-actions on Neural
Networks 10 (3):626-634.
[2] Aapo Hyvärinen, Erkki Oja. 2000. Independent Component Analysis Algorithms and Applications. Neural Networks 13 ((4-5) ):411-430.
[3] Barlow Horace. 1989. Unsupervised learning. Neural Compu-tation (1):295–311.
[4] Bartlett Marian Stewart. 1998. Face image analysis by unsu-pervised learning and redundancy reduction. Ph.D., Univer-sity of California, San
Diego.
[5] Bruce Vicky, Young Andy. 1986. Understanding Face Recognition. British Jounal of Psychology (77):305,327.
[6] Chih-Jen Lin. 2007. Projected Gradient Methods for Non-negative Matrix Factorization. Neural Computation 19:2756-2779.
[7] Draper B., K. Baek, M.S. Bartlett and R. Beveridge. 2003. Recognizing Faces with PCA and ICA. Computer Vision and Image Understanding
Volume 91 (Issues 1-2):115-137
[8] Edelman, S., B. P. Hiles, H. J. Yang, and N. Intrator. 2001. Probabilistic principles in unsupervised learning of visual structure: human data and a
model. Paper read at 15th An-nual Conference on Neural Information Processing Systems (NIPS), Dec 03-08, at Vancouver, Canada.
[9] Edelman, S., N. Intrator, and J. S. Jacobson. 2002. Unsuper-vised learning of visual structure. Paper read at 2nd Interna-tional Workshop on
Biologically Motivated Computer Vi-sion (BMCV 2002), Nov 22-24, at Tubingen, Germany.
[10] Ekman Paul. 1999. Handbook of Cognition and Emotions. John Wiley and Sons Ltd ed. Vol. Facial Expressions.
[1] Lee Daniel, Seung H. Sebastian. 2001. Algorithms for non-negative matrix factorization. Adv. Neural Info in Process-ing Systems 13 556-562.
[11] Ming-Hsuan Yang. 2002. Detecting Faces in Images: a Survey. IEEE Transactions on Pattern Analysis and Machine Intel-ligence 24 (1).
[12] Puga André. 2001. A computational allegory for V1. 2nd Inter-national Symposium on Image and Signal Processing and Analysis at PULA,
CROATIA.
[13] Wendy S. Yambor, Bruce A. Draper and J. Ross Beveridge. 2002. Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection
and Distance Measures. World Sci-entific Press.
[14] Xie Xudong. 2006. Face image analysis and its applications. Ph.D., Hong Kong Polytechnic University (Hong Kong), Hong Kong.
[15] Xue Yun. 2007. Non-negative matrix factorization for face recognition. Ph.D., Hong Kong Baptist University (Hong Kong), Hong Kong.
[16] Zafeiriou, S., A. Tefas, I. Buciu, and I. Pitas. 2005. Class-specific discriminant non-negative matrix factorization for frontal face verification.
Paper read at 3rd International Conference on Advances in Pattern Recognition, Aug 22-25, at Bath, ENGLAND.
[17] Zafeiriou, S., A. Tefas, I. Pitas, and Ieee. 2005. Discriminant NMFFaces for frontal face verification. Paper read at IEEE Workshop on Machine
Learning for Signal Processing (MLSP), Sep 28-30, at Mystic, CT.
[18] Zhao W, Chellappa R., Phillips P. J., Rosenfeld A. 2003. Face recognition: A literature survey. ACM Computing Surveys 35 (4):399-458

5

Matrix mirrors project

More Related Content

What's hot (9)

Viewers also liked (20)

Similar to Matrix mirrors project (20)

More from Renato Roque (9)

Recently uploaded (20)

Matrix mirrors project