1625 signal processing and representation theory

Signal Processing
and
Representation Theory
Lecture 1

Outline:
• Algebra Review
– Numbers
– Groups
– Vector Spaces
– Inner Product Spaces
– Orthogonal / Unitary Operators
• Representation Theory

Algebra Review
Numbers (Reals)
Real numbers, ℝ, are the set of numbers that we
express in decimal notation, possibly with infinite,
non-repeating, precision.

Algebra Review
Numbers (Reals)
Example: π=3.141592653589793238462643383279502884197…
Completeness: If a sequence of real numbers gets
progressively “tighter” then it must converge to a
real number.
Size: The size of a real number a∈ℝ is the square
root of its square norm:
2
aa =

Algebra Review
Numbers (Complexes)
Complex numbers, , are the set of numbers that weℂ
express as a+ib, where a,b∈ andℝ i= .
Example: eiθ
=cosθ+isinθ
1−

Algebra Review
Numbers (Complexes)
Let p(x)=xn
+an-1xn-1
+…+a1x1
+a0 be a polynomial with
ai∈ .ℂ
Algebraic Closure:
p(x) must have a root, x0 in :ℂ
p(x0)=0.

Algebra Review
Numbers (Complexes)
Conjugate: The conjugate of a complex number a+ib
is:
Size: The size of a real number a+ib∈ℂ is the square
root of its square norm:
ibaiba −=+
22
)()( baibaibaiba +=++=+

Algebra Review
Groups
A group G is a set with a composition rule + that
takes two elements of the set and returns another
element, satisfying:
– Asscociativity: (a+b)+c=a+(b+c) for all a,b,c∈G.
– Identity: There exists an identity element 0∈G such
that 0+a=a+0=a for all a∈G.
– Inverse: For every a∈G there exists an element -a∈G
such that a+(-a)=0.
If the group satisfies a+b=b+a for all a,b∈G, then
the group is called commutative or abelian.

Algebra Review
Groups
Examples:
– The integers, under addition, are a commutative group.
– The positive real numbers, under multiplication, are a
commutative group.
– The set of complex numbers without 0, under
multiplication, are a commutative group.
– Real/complex invertible matrices, under multiplication
are a non-commutative group.
– The rotation matrices, under multiplication, are a non-
commutative group. (Except in 2D when they are
commutative)

Algebra Review
(Real) Vector Spaces
A real vector space is a set of objects that can be
added together and scaled by real numbers.
Formally:
A real vector space V is a commutative group with a scaling operator:
(a,v)→av,
a∈ ,ℝ v∈V, such that:
1. 1v=v for all v∈V.
2. a(v+w)=av+aw for all a∈ ,ℝ v,w∈V.
3. (a+b)v=av+bv for all a,b∈ ,ℝ v∈V.
4. (ab)v=a(bv) for all a,b∈ ,ℝ v∈V.

Algebra Review
(Real) Vector Spaces
Examples:
• The set of n-dimensional arrays with real coefficients is a
vector space.
• The set of mxn matrices with real entries is a vector
space.
• The sets of real-valued functions defined in 1D, 2D, 3D,
… are all vector spaces.
• The sets of real-valued functions defined on the circle,
disk, sphere, ball,… are all vector spaces.
• Etc.

Algebra Review
(Complex) Vector Spaces
A complex vector space is a set of objects that can be
added together and scaled by complex numbers.
Formally:
A complex vector space V is a commutative group with a scaling operator:
(a,v)→av,
a∈ ,ℂ v∈V, such that:
1. 1v=v for all v∈V.
2. a(v+w)=av+aw for all a∈ ,ℂ v,w∈V.
3. (a+b)v=av+bv for all a,b∈ ,ℂ v∈V.
4. (ab)v=a(bv) for all a,b∈ ,ℂ v∈V.

Algebra Review
(Complex) Vector Spaces
Examples:
• The set of n-dimensional arrays with complex coefficients
is a vector space.
• The set of mxn matrices with complex entries is a vector
space.
• The sets of complex-valued functions defined in 1D, 2D,
3D,… are all vector spaces.
• The sets of complex-valued functions defined on the
circle, disk, sphere, ball,… are all vector spaces.
• Etc.

Algebra Review
(Real) Inner Product Spaces
A real inner product space is a real vector space V
with a mapping 〈V,V〉→ℝ that takes a pair of vectors
and returns a real number, satisfying:
〈u,v+w〉= 〈u,v〉+ 〈u,w〉 for all u,v,w∈V.
〈αu,v〉=α〈u,v〉 for all u,v∈V and all α∈ℝ.
〈u,v〉= 〈v,u〉 for all u,v∈V.
〈v,v〉≥0 for all v∈V, and 〈v,v〉=0 if and only if v=0.

Algebra Review
Examples:
– The space of n-dimensional arrays with real
coefficients is an inner product space.
If v=(v1,…,vn) and w=(w1,…,wn) then:
〈v,w〉=v1w1+…+vnwn
– If M is a symmetric matrix (M=Mt
) whose eigen-
values are all positive, then the space of n-
dimensional arrays with real coefficients is an inner
product space.
〈v,w〉M=vMwt

Algebra Review
Examples:
– The space of mxn matrices with real coefficients is an
inner product space.
If M and N are two mxn matrices then:
〈M,N〉=Trace(Mt
N)

Algebra Review
Examples:
– The spaces of real-valued functions defined in 1D,
2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:
– The spaces of real-valued functions defined on the
circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
∫
∞
∞−
= dxxgxfgf )()(,
∫=
π
θθθ
2
0
)()(, dgfgf

Algebra Review
(Complex) Inner Product Spaces
A complex inner product space is a complex vector
space V with a mapping 〈V,V〉→ℂ that takes a pair of
vectors and returns a complex number, satisfying:
〈u,v+w〉= 〈u,v〉+ 〈u,w〉 for all u,v,w∈V.
〈αu,v〉=α〈u,v〉 for all u,v∈V and all α∈ℝ.
– for all u,v∈V.
〈v,v〉≥0 for all v∈V, and 〈v,v〉=0 if and only if v=0.
uv,vu, =

Algebra Review
Examples:
– The space of n-dimensional arrays with complex
coefficients is an inner product space.
– If M is a conjugate symmetric matrix ( ) whose
eigen-values are all positive, then the space of n-
dimensional arrays with complex coefficients is an
〈v,w〉M=vMwt
nn11 wv...wvwv, ++=
t
MM =

Algebra Review
Examples:
– The space of mxn matrices with real coefficients is an
If M and N are two mxn matrices then:
( )NMNM, t
Trace=

Algebra Review
Examples:
– The spaces of complex-valued functions defined in
1D, 2D, 3D,… are real inner product space.
If f and g are two functions in 1D, then:
– The spaces of real-valued functions defined on the
circle, disk, sphere, ball,… are real inner product
spaces.
If f and g are two functions defined on the circle, then:
∫
∞
∞−
= dxxgxfgf )()(,
∫=
π
θθθ
2
0
)()(, dgfgf

Algebra Review
Inner Product Spaces
If V1,V2⊂V, then V is the direct sum of subspaces V1,
V2, written V=V1⊕V2, if:
– Every vector v∈V can be written uniquely as:
for some vectors v1∈V1 and v2∈V2.
21 vvv +=

Algebra Review
Inner Product Spaces
Example:
If V is the vector space of 4-dimensional arrays, then
V is the direct sum of the vector spaces V1,V2⊂V
where:
– V1=(x1,x2,0,0)
– V2=(0,0,x3,x4)

Algebra Review
Orthogonal / Unitary Operators
If V is a real / complex inner product space, then a
linear map A:V→V is orthogonal / unitary if it
preserves the inner product:
〈v,w〉= 〈Av,Aw〉
for all v,w∈V.

Algebra Review
Examples:
– If V is the space of real, two-dimensional, vectors and
A is any rotation or reflection, then A is orthogonal.
A=
v2
v1
A(v2)
A(v1)

Algebra Review
Examples:
– If V is the space of real, three-dimensional, vectors
and A is any rotation or reflection, then A is
orthogonal.
A=

Algebra Review
Examples:
– If V is the space of functions defined in 1D and A is
any translation, then A is orthogonal.
A=

Algebra Review
Examples:
– If V is the space of functions defined on a circle and A
is any rotation or reflection, then A is orthogonal.
A=

Algebra Review
Examples:
– If V is the space of functions defined on a sphere and
A is any rotation or reflection, then A is orthogonal.
A=

Outline:
• Algebra Review
• Representation Theory
– Orthogonal / Unitary Representations
– Irreducible Representations
– Why Do We Care?

Orthogonal / Unitary Representation
An orthogonal / unitary representation of a group G
onto an inner product space V is a map Φ that sends
every element of G to an orthogonal / unitary
transformation, subject to the conditions:
1. Φ(0)v=v, for all v∈V, where 0 is the identity element.
2. Φ(gh)v=Φ(g) Φ(h)v

Examples:
– If G is any group and V is any vector space, then:
is an orthogonal / unitary representation.
– If G is the group of rotations and reflections and V is
any vector space, then:
vvg =Φ )(
vgvg )det()( =Φ

Examples:
– If G is the group of nxn orthogonal / unitary
matrices, and V is the space of n-dimensional arrays,
then:
( )vgvg =Φ )(

Examples:
– If G is the group of 2x2 rotation matrices, and V is
the vector space of 4-dimensional real / complex
arrays, then:
( ) ( )),(),,(,,,)( 43214321 xxgxxgxxxxg =Φ

Irreducible Representations
A representation Φ, of a group G onto a vector space
V is irreducible if cannot be broken up into smaller
representation spaces.
That is, if there exist W⊂V such that:
Φ(G)W⊂W
Then either W=V or W=∅.

If W⊂V is a sub-representation of G, and W⊥
is the
space of vectors perpendicular to W:
〈v,w〉=0
for all v∈W⊥
and w∈W, then V=W⊕W⊥
and W⊥
is also
a sub-representation of V.
For any g∈G, v∈W⊥
, and w∈W, we have:
So if a representation is reducible, it can be broken
up into the direct sum of two sub-representations.
( ) ( ) ( ) ( ) ( ) wvgwggvgwgv ,,,0 11
ΦΦΦΦ=Φ= −−

Examples:
– If G is any group and V is any vector space with
dimension larger than one, then:
is not an irreducible representation.
vvg =Φ )(

Examples:
– If G is the group of 2x2 rotation matrices, and V is
the vector space of 4-dimensional real / complex
arrays, then:
is not an irreducible representation since it maps the
space W=(x1,x2,0,0) back into itself.
( ) ( )),(),,(,,,)( 43214321 xxgxxgxxxxg =Φ

Why do we care?

Why we care
In shape matching we have to deal with the fact that
rotations do not change the shape of a model.
=

Exhaustive Search
If vM is a spherical function representing model M
and vn is a spherical function representing model N,
we want to find the minimum over all rotations T of
the equation:
( )
( )NMNM
NMNM
vTvvv
vTvvvT
,2
),,(
22
2
−+=
−=D

Exhaustive Search
If V is the space of spherical functions then we can
consider the representation of the group of rotations
on this space.
By decomposing V into a direct sum of its
irreducible representations, we get a better
framework for finding the best rotation.

Exhaustive Search (Brute Force)
Suppose that {v1,…,vn} is some orthogonal basis for
V, then we can express the shape descriptors in terms
of this basis:
vM=a1v1+…+anvn
vN=b1v1+…+bnvn

Then the dot-product of M and N at a rotation T is
equal to:
( )
( )
( )∑
∑∑
∑∑
=
==
==
=
=








=
n
ji
jiji
n
j
jj
n
i
ii
n
j
jj
n
i
iiNM
vTvba
vTbva
vbTvavTv
1,
11
11
,
,
,,

( ) ( )∑=
=
n
ji
jijiNM vTvbavTv
1,
,,
So that the nxn cross-multiplications are needed:
T(vn)
vM
v1
v2
vn
=
+
+
+
T(v1)
=
+
+
+
T(v2)
T(vN)
…
…

Exhaustive Search (w/ Rep. Theory)
Now suppose that we can decompose V into a
collection of one-dimensional representations.
That is, there exists an orthogonal basis {w1,…,wn} of
functions such that T(wi)∈wiℂ for all rotations T and
hence:
〈wi,T(wj)〉=0 for all i≠j.

Then we can express the shape descriptors in terms
of this basis:
vM=α1w1+…+αnwn
vN=β1w1+…+βnwn

And the dot-product of M and N at a rotation T is
equal to:
( )
( )
( )
( )∑
∑
∑∑
∑∑
=
=
==
==
=
=
=








=
n
i
iiii
n
ji
jiji
n
j
jj
n
i
ii
n
j
jj
n
i
iiNM
wTw
wTw
wTw
wTwvTv
1
1,
11
11
,
,
,
,,
βα
βα
βα
βα

( ) ( )∑=
=
n
i
iiiiNM wTwvTv
1
,, βα
So that only n multiplications are needed:
T(wn)
vM
w1
w2
wn
=
+
+
+
T(w1)
=
+
+
+
T(w2)
T(vN)
…
…

1625 signal processing and representation theory

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to 1625 signal processing and representation theory (20)

More from Dr Fereidoun Dejahang (20)

Recently uploaded (20)

1625 signal processing and representation theory