PDF for linear_algebra_1. Including various concepts of linear algebra.

Inverse of A (A) for square matrices
A
*
A =
l =
AA-
*
Identity --
Axcolj ofAlcoljof [
[I] / ?]=
(0i]↑
A At I -like solving
Wi
and Ax[s]=7%)
two systems
of
equations

Gauss-Jordan (Solve two
eqs.
at once)
( =](5] =
[0]
I
I
(t)Toheart.
aotininae

Let's check
[37=
(i=7= (i)
At A
I
for example : -2x3 +1x7 =
- +
7=
1

Transpose of A (AY
A =
(5) -
=(=]nxM
ma
A =
[0/ x] -At [apj]
first row of A -- first column of At
second row of A-second column of At
&
⑧

Inverse of AB (if we have A and B
(AB) (B At) =
I Because A (BBY Al (AC) A =
AA
*
-
I
what is the inverse of AT? (A
AA
*
=
I
I
Answer
-
This is (At

Ex A =
U
(9)(817=
[8s) How to convert Ez,
A=U to A=
bu
A =el
=
Ei)
d
U stands for upper triangular
que
[81):
(ii) [8's]
- stands for lower
triangular
u has pirots on the diagonal
Pue
I has ones on the diagonal

A =
L U
(81)=
(i) [03]
I'93 (83] [ ]
↳ ↓
for Pirots

A. =
EA= (e =
(e)
I
g
-
---
A =
U
L -
U
No row exchange
En
we have a nice matrix

Esti,
A =
4 = A =
?- ?
&
"U
A =
!Eas why is this form nicer than the other forms
i
A = L
-
Answer:
Because if no row exchanges,
multipliers go directly into L

PDF for linear_algebra_1. Including various concepts of linear algebra.

Learn & Earn
A =
(3 %]
Question :
Decompose A into Land H

(1) [ =]=
(0)
E21 A U
Sit
(ii) ilsi)
En,
I IE! =
L

Learn & Earn
A- Tee !I
Question :
Decompose A into Land U

All ) (e)
n
I - =
/08I
eftoroestadosorthe ↳
0 11

Permutation P : execute row
exchanges
(id] (:] [a]
ps
exchangeato Ripp
- I
I
A =
Lu without vow
exchange
PA =
LH with row
exchange

symmetric matrix
25] AT A
L
/
is it symmetric?
(23] =
= i]
↑T R
Fri
always symmetric

(i) ( =]
3 x2
why ? Take transpose
(RR) =
RT RTT=
RYR
wa
wa

Vector
spaces and subspaces
-
column space of a matrix
I Nullspace of a matrix
m e
n e e
Vector space requirements :
V+ W and CV are in the space
db ↓ Y
rector
rector vector constant
↑ &
All combinations CV t
dw are in the space
constant
[ ↳ constant

&
mini vector space ?
It's a bunch of rectors that we can add
any two vectors
in the
space
and the answer stays in the space ; or
we can
multiply any vector in the
space by any
constant
and the answer
stays in the space ; or we combine the two
previous sentences into one,
that means all linear combinations
of
any
two vectors stay in the space

subspaces : some vectors inside the
given space
that still make up a vector space of their own
A vector
space inside a vector
space
* ↓
Plane through (8) 3
P L is a subspace of R
⑧
W
line through (8) is a -
subspace of R3

etion
:
suppose I take two subspaces,
like Pand L ,
and
put them together (take their union),
is that a subspace ?
#L = all vectors in Por L or
both
↓
This (is) (is nott a subspace

Question : How about their intersection ?
Pl = all vectors in both P and L
e
n
6This (is) (is not
a subspace
#
:
-
8
8.
↓ P
-

subspaces S and T
: Intersection SIT
Is a
subspace
let's
say
and W are two rectors in ST
that means they are both in S ;
also both in I
sum of two vectors
v + W is in S &
WeW is
int]-VIW is in S1TV Req 9
scalar multiplication ofa vector
cis
ins]-> cr is in S1T/REq. 2
cV is in T

Vector
spaces and subspaces
↓
column space of a matrixe
Nullspace of a matrix

comyspace
of A is a
subspace of RP <(A)
4x 3
A-
[] CCA) : all linear combinations of columns
How big is <(A) space? Does it fill the full 11 space ?
or is it a
subspace inside ?

A
linear combinations of columns
Does Ax=b have a solution for
every b ?
(a)()-(
A x
-
which b's allow this system to be solved?
tell me one
right-hand side (b) that we know we can
solve it

-
↳- ot
x ,
+
x2 +
2x, = 1
↑ of firstcol+0 of 2nd +O of 3rd
2x ,
+
x2 +
3xz = 2
can
you
solve it in 5 see ?
3x1 +
x2 +
4xz =
3
kx,
+
x2 +
5x = 4

↳( ) (i)
one
way to find b is to think of solution first,
then see
what b turns out to be
(i)(i)=(b)

which b's allow this system to be solved?
we can solve Ax= b
whenb is a combination of the columns in A
in other words,
when bis in the race of A
((A)
u e
tells us when we can solve Ar=b

I
can't throw awray any
columns and have the same
column
space ?
(Yes) (NO)

I
which one do
you suggest
I throw
away ?

pot
columns
I
me
do
you suggest
Ithrow
away ? Columns
Because it's the combination of column and I
so it contribute
nothing new

I
could I have thrown
away column 1 ?
convention for pivot columns :
start from the first column

[23wr
space
of A
:twoDimensione

Nullspace of A =
all solutions x=
[ ) to Ax =
O
C(i) =
(8)
This nullspace is a subspace of RB
Aman
In our
example,
column space was in R*

Nullspace
of A NCA)
C(i) =
(8)
tell me one solution
very quickly ?

Nullspace
of A NCA)
contains
[8], 13
& try to find it by
looking at combinations
C(i) =
(8) of columns

Nullspace
of A NCA)
(i)
contain
(i)
↑
In)():
(8) Lindy

check that solutions to Ax=
O always give a
subspace
& If Av=O and Aw=0 then A(V+w) =
0
AV+Aw =
0
any scalar 0 + 0 =
0 ~
& If Av=0 ,
then A(12v) =
8 - 12 Ar =Ow
&
-
minilaw
:
ACB+C) = AB +AC

many
so far
(two ways
to construct subspaces)
vectors
column space : I tell
you
a few columns and
say take
their combinations
Nullspace : I didn't tell you
what's in it .
We have to figure it out
I just told the system of eqs that I have to
satisfy

computing the nullspace (Ax= 0)
Pivot variables -
free variables
special solutions

A =
TheI
let's do nation
now extended
to rectangular Case where
↑ we have to continue
even if there's zero
How many equations ? In the pirot position
How
many unknowns ?

a
first pirot
1 2 2
~ -
! ?
A =
[2 I I 3 ~
O 02 &
-it's O and no hope for
more- )=
row exchange
2 ↓
->
I so ,
nothing to do
0000 on 2nd column
pivot columns &*** free columns
Rank of A =
Number of Pivots =2

22 , 22 Els 2
I can
assign anything I like
P?
]=
u
Se
0000
5 to se
,
and
q
*** free columns
pivot columns
nullspace
O
↓ Vector besome more rectors
in the nulspace ?
x,
+
2x2 +
2xz +
2xp =
0
2xz +
1x = 0 =
TimeAu
- 2xc011 +
xc0l2 =
0

more vectors
->
Is it the whole nullspace?
scalar2] in nullspe e
we have two free variables
-
2,
+22 -
See
=
Time
1)
xq=
1 what are all
solutions to Ax= 0
or Ux =
0 ?
calor[02)
some more reco is
/
in nulespace
& or the whole nullspace

f)
ie
the nullspace contains
e all the combinations of
the special solutions
There is one special solution for every free variable
For Amani (# free variables)=
n-rank of A

I [ I
At
The 8 sination = e
& Zero row in U means
original row in A was a combination
of other rows
& U is in vow echelon form
-
↳ staircase pattern with zeros below

Reeced row echelon form &Each pivot is the only
non-zero
entry in its column
u =
! -] & Pivots should be
changed to
1 by row
operations
2 prot.
120
= =
R
L I
% 02
I
·
02a)
000 000f

original matrix row echelon reduced row echelon
I!
I
I
?
↑
Cor
30 8 10
A U R
x, +
2x2 +
2x, +
2x =
0 x,
+
2x2 +
2x, +
2xp =
0x +
2x2 -
2xp =
0
-2
2x, +1x2 + Wx
,
+
&x =
0
2xz +
1x = 0
xz1 xq
= 0
3x,
+
Wx2+
82
,
+
104 =
0
Ax = 0 Hx =
8 Rx = 0
& ↑ ↑
All have the same solutions

pivot columns
↓ d
x r
I pirti cols
free
0008
-2 -
Pivot Var
&
2
A ->
free var & O
11
O -> pivot var e -
2
11 Matrix tion*[Ieirotar
O -> free var - I
↑
special solution

let's do another example :
12 3 # 2 3
A:
(8) -i000
I IO &
2
I
O 22 -
00 0 -U
04 4 8 O O
↳ -
Rank= 2
again
pirations ↑
free col
as =
(i)-mepac]
-I of coll A
->
of col2 A
#
O13 A

I I 0
·
-F
-
00 O
L
I
I Matrix tion*[Ieirotar
O 00
U R
a
=fil

Complete solution of Ax=b
Matrice rank (r) and number of solutions

x,
+
2xz +
2k, +
2x =
b ,
2x,
+
(x +
0x
,
+
8xg =
bz
22 2
I it
30 8 18
3x , +8xz +
8x
, +loup= by
Augmented matrix =
(A b]
L
1222 b,
I I
1222 b ,
I netsb()
->
O 0 2 + be -2b, -> 0024 be-2b,
0024
b -
3b , 00 0 0
by-b-b,
last row ->O=by-be-b,
-b=
be +
b,

ity
condition on b
& column space
Ax =
b solvable when b is in <(A)
If a combination of rows of A gives zero row,
then same combination
of entries of b must
give O .

To find complete solutions to Ax=b x2 =
0
- R =
D
&
particular
: Set all free variables to zero .
apirot
solve b for pivot variables
⑪ 2422
I
I
↳ x
, + 2xz=
1 x, =
-
2
000)
&
Knullspace
223 =3
-
3,
=
30
-
x =
xp +
2n reason
↑
complete solution
-

Ito(i)=(i)
(i)agogonfocapeetesolutions
Sparticula nullspace

"complete+forica
Answer :
No,
it does Not
go through the
origin
[i]

Wi
lecture
* Matric rank (r) and number of solutions
A Linear independence
*
Spanning a
space
* Basis and dimension
* Four fundamental subspaces (for matrix Al

m
by n matrixe A of rank
number of pivots
① m> n & n > m & =
m (square)
ES [= =
=] [==]
ran r2m r1n(=
m)
Full column rank Full row rank Full rank
r=
n r=
m r=
n =
m

Full column rank (r=n) Example
n pivot variables
->
(izz()
It has
E A=
/,] =
Li
0 free variables
S
↓Ax=
8 only solution
Nullspace of A N(A) =
(zero rectory
I only one
solution to Az=b : x=
Up (unique solution ifit exists)
10 or 1 solution) E
means

Full row rank (r=
m)
m pivot variables can solve An=b for everyb
It has
E
>
n - M free variables
Example
A =
b : :)-
Because No zero rows
in R

Full rank (r=m= n)
Example
A=
(si) - R =
t
Nullspace of A =
Szero rector]
solution to Az=b : can solve for everyb,
an
learnique solution
I
because r=
m because = n

Big picture
u=
m =
nu=
n <m r=
m<n ram , ven
R = I R =
(0) R =
(i +] R =
(86]
1 solution to 0 or solution & solutions 0 or 0 solutions
As =
b to Ax=b to Ax= b to Ax= b
The rank tells
you everything about the number of solutions

-> Linear independence
spanning a
space
Basis and dimension
Four fundamental subspaces (for matrix As

Review
Suppose A is
mbyn with
men
-
Then there are nonzero solutions to An= 0
e n
,
- more unknowns than equations
Reason :
there will be free variables
n variables,
and most m pivots
so at least,
there will be n-m free variables
-

Independence (for bunch of vectors
S
-
short for
linearly independent
- I
Vectors V,, VI . ... , n are independent if combining by just multiplying
and adding
no combination gives zero vector (except the zero combination)
C, 1, +2zV2+ . . . +
2nVnFO (except all <i =
0)
vr v
If a combination
gives zero vector-> i ....
n
are dependent

Example
=2
->
2V, +(1) V2 =
0 - dependent
a
->
OV,
+Vz
=
0 - dependent
or
any
If
any
V
,
=
0,
then vectors V, VI , ..., Un are dependent

Exampl
n
- hand re are independent
V, Ve ,
Y
,
are dependent
En -A=
)*(()=(03
~
↳
Hismbynwithmonsto A e

Independence (repeat the definition using null
space)
when Vi , Ve ,
.... Un are columns of A
,
solutions to Ax=
0
no free variables
-
They are ependent if nullspace of A is
only [Vector] ant=n
They are dependent if Ac =
0 for some nonzero c -> rank u
- -
↓
some free variables

Linear independence
->
spanning a
space
-> Basis and dimension

Vectors V. VI s ...,
Ye sn a
space means : the space consists
of all linear combinations of those vectors
Example : the columns of a matrix
span the column space
Basis a
sequence of rectors VisVas ...,
V
I
with
two properties : & They are
independent
& They span the
space

Example :
find basis for R space
(8)+
2()+(i)=(8)
one basis is
(7,
(8):
(i) ↑
only for , = = =
0 independent
For R": n vectors gives basis if
the nxu matrix with those columns
Another basis is
!!):
(5):
(E) is full rank (invertible)
↑ enough
forR?? No
For a
given space,
every basis for
put them in a matrix,
do elimination the space has same number of vectors
All columns are
pirot cel ? Yes-basis ~ Dimension f
the space

Example :
find basis for CCA) pirot columns
A =
fi] [,
a
lisis
a
lisis
↑ 4
pivot columns
↑
2 =
rank (A) =
number of pivot columns =
dimension of ((A)

Example :
find basis for N(A) -> special solutions
-
A =
fi? " Juan (t (i
pivot columns
-
# of cols
dimension of NCA) =
number of free variables =*
rank
u - r
dimension of <(A) =
~

Linear independence
spanning a
space
Basis and dimension
->

↑ subspaces of matrix Amxn
& column space ((A)
② nullspace N(A)
③ row space =
all combinations of all combinations of =
((AY)
rows ofA columns of At
⑧ nullspace of AT =
NCAT
-
called left nulspace

↑ subspaces of matrix Amen
subspace of
-
mension
& column space ((A) RM L
② nullspace N(A) RY n -
r
③ row space ((AY) RY fr
⑧ left nullspace NCAT RM m - r

PDF for linear_algebra_1. Including various concepts of linear algebra.

More Related Content

Similar to PDF for linear_algebra_1. Including various concepts of linear algebra. (20)

Recently uploaded (20)

PDF for linear_algebra_1. Including various concepts of linear algebra.