CHapter One -Basic_Probability_Theory.ppt

Chapter 1: Introduction to Probability Theory
Ambo University
Hachalu Hundessa Campus
School of Electrical Engineering and Computing
Department of Electrical & Computer Engineering
Probability and Random Process (ECEg-2114)
Lecturer: Getahun Shanko(MSc.)
Contact address: getahunshanko605@gmail.com
Ambo, Ethiopia 2023

Basic Concepts of Probability Theory
Outline
 Introduction
 Sample Space and Events
 Basic Set Operations
 Axioms and Properties of Probability
 Conditional Probability
 Independence of Events
2

Introduction
 Probability is a measure of the chance of occurrence that an
event will occur.
 It deals with the chance of occurring any particular
phenomena.
 Probability is the study of randomness and uncertainty.
 The different types of events in probability are:
• Random-the event may or may not occur
• Certain- the occurrence of the event is
inevitable(something that is sure to happen)
• Impossible-the event will never occur
3

Sample Space and Events
i. Random Experiment
 A random experiment is an experiment in which the outcome
varies in an unpredictable manner when the experiment is
repeated under the same conditions.
Examples:
• Tossing a coin
• Rolling a die
• Transistor life time
ii. Sample Space
 The sample space is the set of all possible outcomes of a random
experiment.
4

Sample Space and Events Cont’d….
 The sample space is denoted by Ω and the possible outcomes
are represented by
iii. Event
 An event is any subset of the sample space
 Events can be represented by A, B, C,.……
Example-1:
Consider a random experiment of rolling a die once.
i. Sample Space
}
......
,
,
{ n
2
1 




i

}
6
5,
4,
3,
2,
,
1
{


5

ii. Some possible events
 An event of obtaining even numbers
 An event of obtaining numbers less than 4
Example-2:
Consider a random experiment of flipping a fair coin twice.
i. Sample space
}
6
4,
,
2
{

A
}
3
2,
,
1
{

B
}
TT
TH,
HT,
,
{HH


6

ii. Some possible events
 An event of getting exactly one head
 An event of getting at least one tail
 An event of getting at most one tail
}
TH
,
{HT
A 
}
TT
TH,
,
{HT
B 
}
TH
,
,
{ HT
HH
C 
7

Discrete and Continuous Sample space
• A sample space Ω is said to be discrete if it consists of a finite
number of sample points.
E.g. Find the sample space for the experiment of tossing a coin once.
Ω ={H, T}
• A sample space Ω is said to be continuous if the sample points
constitute a continuum.
E.g. Find the sample space for the experiment of measuring (in
hours) the lifetime of a transistor.
Clearly all possible outcomes are nonnegative real numbers.
That is, Ω ={τ: 0≤ τ ≤ ∞} where τ represents the life of a
transistor in hours.
8

Basic Set Operations
 We can combine events using set operations to obtain other
events.
1. Union
The union of two events A and B is defined as the set of outcomes
that are either in A or B or both and is denoted by .
B
A 
}
:
{
}
or
:
{ B
A
B
A
B
A
B
A 






 




 

E F
Ω
A B
AB
9

Basic Set Operations Cont’d…..
2. Intersection
The intersection of two events A and B is defined as the set of
outcomes that are common to both A and B and is denoted by
.
B
A 
}
:
{
}
and
:
{ B
A
B
A
B
A
B
A 






 




 

A B
Ω
AB
10

3. Complement
The complement of an event A is defined as the set of all outcomes
that are not in A and is denoted by .
A
}
:
{
}
and
:
{ A
A
A
A 








 





E
Ω
A A
11

4. Mutually Exclusive (Disjoint) Events
Two events A and B are said to be mutually exclusive or disjoint
if A and B have no elements in common, i.e.,
5. Equal Events
Two events A and B are said to equal if they contain the same
outcomes and is denoted by A=B.
B
A


 B
A
12


 B
A

Some Properties of Set Operations
1. Elementary Properties
2. Commutative Properties
3. Associative Properties
A
A
iv
A
A
vii
A
iii
A
A
vi
ii
A
A
v
i




















.
.
.
.
.
.
.
A
B
B
A
A
B
B
A






C
B
A
C
B
A
C
B
A
C
B
A








)
(
)
(
)
(
)
(


13

Some Properties of Set Operations Cont’d…..
4. Distributive Properties
5. De Morgan's Rules
 The union and intersection operations can be repeated for
an arbitrary number of events as follows.
)
(
)
(
)
(
)
(
)
(
)
(
C
A
B
A
C
B
A
C
A
B
A
C
B
A












B
A
B
A
B
A
B
A






)
(
)
(








n
i
n
i
n
n
i
i
A
A
A
A
A
A
A
A
1
2
1
2
1
1
...
....




14

Axioms and Properties of Probability
 Probability is a rule that assigns a number to each event A in the
sample space, Ω.
 In short , the probability of any event A is given by
space
sample
in the
elements
of
number
the
is
-
)
(
event
in the
elements
of
number
the
is
-
)
(
where
)
(
)
(
)
(




n
A
A
n
n
A
n
A
P
15

Axioms and Properties of Probability Cont’d…..
 The probability of an event A is a real number which satisfies the
following axioms.
1. Probability is a non-negative number, i.e.,
2. Probability of the whole set is unity, i.e.,
From axioms (1) and (2), we obtain
0
)
( 
A
P
1
)
( 

P
1
)
(
0 
 A
P
16

3. Probability of the union of two mutually exclusive events is the sum
of the probability of the events, i.e.,
 We can generalize axiom (3) for n pairwise mutually exclusive
(disjoint) events.
 If A1, A2, A3, …, An is a sequence of n pairwise mutually exclusive
(disjoint) events in the sample space Ω such that
)
(
)
(
)
(
then
,
If B
P
A
P
B
A
P
B
A 


 

then
,
for
, j
i
A
A j
i 



)
(
1
1











 n
i
i
n
i
i A
P
A
P 
17

 By using the above probability axioms, other useful
properties of probability can be obtained.
Proof:
)
(
1
)
(
.
1 A
P
A
P 

)
(
1
)
(
1
)
(
,
)
(
)
(
1
)
(
)
(
,
)
(
)
(
)
(
but
,
)
(
)
(
)
(
A
P
A
P
P
A
P
A
P
A
A
P
P
A
P
A
P
P
A
A
A
P
A
P
A
A
P
A
A

























18

 We can decompose the events A, B and AUB as unions of
mutually exclusive (disjoint) events as follows.
B
A B
A B
A
A B
events.
disjoint
are
A
and
,
that
see
can
We B
B
A
B
A 


19

 From the above Venn diagram, we can write the following
relations.
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
.
B
A
P
A
P
B
A
P
B
A
P
B
A
P
A
P
B
A
B
A
A
i














)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
.
B
A
P
B
P
B
A
P
B
A
P
B
A
P
B
P
B
A
B
A
B
ii














)
(
)
(
)
(
)
(
.
B
A
P
A
P
B
A
P
B
A
A
B
A
iii








 )
(
)
(
)
(
)
(
.
B
A
P
B
P
B
A
P
B
A
B
B
A
iv









)
(
)
(
)
(
)
(
)
(
)
(
)
(
.
B
A
P
B
A
P
B
A
P
B
A
P
B
A
B
A
B
A
B
A
v















20

Proof:
 We can generalize the above property for three events A, B and C
as follows.
)
(
)
(
)
(
)
(
.
2 B
A
P
B
P
A
P
B
A
P 
 


)
(
)
(
)
(
)
(
)
(
)
(
)
(
But,
)
(
)
(
)
(
B
A
P
B
P
A
P
B
A
P
B
A
P
B
P
B
A
P
B
A
P
A
P
B
A
P














)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
C
B
A
P
C
B
P
C
A
P
B
A
P
C
P
B
P
A
P
C
B
A
P














21

 For n events A1, A2, A3,…,An the above property can be
generalized as:
Proof:
)
...
(
)
1
(
....
)
(
)
( 2
1
1
1
n
n
n
k
j
k
j
n
j
j
n
i
i A
A
A
P
A
A
P
A
P
A
P 



 













 


)
(
)
(
)
(
.
3 B
P
A
P
B
A
P 


)
(
)
(
)
(
0
)
(
But,
)
(
)
(
)
(
)
(
B
P
A
P
B
A
P
B
A
P
B
A
P
B
P
A
P
B
A
P











22

Axioms and Properties of Probability Cont’......
Example-1:
A box contains 10 identical balls numbered 0, 1, 2,…,9. A single
ball is selected from the box at random. Consider the following
events.
A: number of ball selected is odd
B: number of ball selected is multiple of 3
C: number of ball selected is less than 5
Find the following probabilities.
)
(
c.
)
(
e.
)
(
b.
)
(
d.
)
(
a.
C
P
C
B
A
P
B
P
B
A
P
A
P



23

Solution:
 The sample space and the events are given by:
 The number of elements in the sample space and events are:
}
9
7,
6,
5,
4,
3,
2,
1,
0,
{
}
9
6,
3,
{
}
9
3,
{
}
9
7,
5,
3,
1,
{
}
4
3,
2,
1,
0,
{
}
9
8,
7,
6,
5,
4,
3,
2,
1,
0,
{







C
B
A
B
B
A
A
C



9
)
(
3
)
(
2
)
(
5
)
(
5
)
(
10
)
(







C
B
A
n
B
n
B
A
n
A
n
C
n
n



24

Thus, probabilities of the given events are given by:
Example-2: 2
1
10
5
)
(
)
(
)
(
.
10
9
)
(
)
(
)
(
.
10
3
)
(
)
(
)
(
.
5
1
10
2
)
(
)
(
)
(
.
2
1
10
5
)
(
)
(
)
(
.


















n
C
n
C
P
c
n
C
B
A
n
C
B
A
P
e
n
B
n
B
P
b
n
B
A
n
B
A
P
d
n
A
n
A
P
a






)
(
.
)
(
.
)
(
.
)
(
.
)
(
.
)
(
.
:
find
,
75
.
0
)
(
and
8
.
0
)
(
,
9
.
0
)
(
Given
B
P
f
B
A
P
d
B
A
P
b
B
A
P
e
B
A
P
c
B
A
P
a
B
A
P
B
P
A
P





 


25

Solution:
b. P(A⋂B) = P(A)- P(A ⋂B)
P(A⋂B) = 0.9-0.75 =0.15
95
.
0
)
(
75
.
0
8
.
0
9
.
0
)
(
)
(
)
(
)
(
)
(
.









B
A
P
B
A
P
B
A
P
B
P
A
P
B
A
P
a




05
.
0
)
(
95
.
0
1
)
(
)
(
1
)
(
)
(
.








B
A
P
B
A
P
B
A
P
B
A
P
B
A
P
c





25
.
0
)
(
75
.
0
1
)
(
)
(
1
)
(
)
(
.








B
A
P
B
A
P
B
A
P
B
A
P
B
A
P
d





85
.
0
)
(
75
.
0
9
.
0
1
)
(
)
(
)
(
1
)
(
.









B
A
P
B
A
P
B
A
P
A
P
B
A
P
e




2
.
0
)
(
8
.
0
1
)
(
)
(
1
)
(
.







B
P
B
P
B
P
B
P
f
26

Exercise:
7
.
0
)
(
that
show
then
,
8
.
0
)
(
and
9
.
0
)
(
If
.
4
.
1
)
(
that
show
then
,
1
)
(
)
(
If
.
3
.
0
)]
(
)
[(
that
show
then
,
)
(
)
(
)
(
If
.
2
).
(
)
(
that
show
then
,
If
.
1












B
A
P
B
P
A
P
B
A
P
B
P
A
P
B
A
B
A
P
B
A
P
B
P
A
P
B
P
A
P
B
A







27

Conditional Probability
(1)
0
)
(
,
)
(
)
(
)
/
( 
 B
P
B
P
B
A
P
B
A
P

(3)
)
(
)
/
(
)
(
)
/
(
)
( A
P
A
B
P
B
P
B
A
P
B
A
P 


28

Conditional Probability Cont’d…….
 Then using equation (3), we will get
 We know that
 Substituting equation (5) into equation (4), we will get
(4)
)
(
)
(
)
/
(
)
/
(
OR
)
(
)
(
)
/
(
)
/
(
A
P
B
P
B
A
P
A
B
P
B
P
A
P
A
B
P
B
A
P 

(5)
)
(
)
/
(
)
(
)
/
(
)
(
B)
(
)
(
)
(
A
P
A
B
P
A
P
A
B
P
B
P
A
P
B
A
P
B
P




 

(6)
)
(
)
/
(
)
(
)
/
(
)
(
)
/
(
)
/
(
A
P
A
B
P
A
P
A
B
P
A
P
A
B
P
B
A
P


29

 Similarly,
 Equations (6) and (7) are known as Baye’s Rule.
 Baye’s Rule can be extended for n events as follows.
 Let events A1, A2, A3, …, An be pairwise mutually exclusive (disjoint ) events and their union be the sample space Ω, i.e.
(7)
)
(
)
/
(
)
(
)
/
(
)
(
)
/
(
)
/
(
B
P
B
A
P
B
P
B
A
P
B
P
B
A
P
A
B
P




















n
i
i
n
i
i
n
i
i
j
i
A
P
A
P
A
A
A
1
1
1
)
(
and



30

 Let B be any event in Ω as shown below.
1
A
3
A
2
A
1

n
A
n
A
B
.....
.....








)
(
)
(
But,
)
(
...
)
(
)
(
)
....
(
2
1
2
1
j
i
j
i
n
n
A
B
A
B
A
A
A
B
A
B
A
B
B
A
A
A
B
B














31

 The events are mutually exclusive events.
 In short,
 Then using equation (4), we will obtain
(8)
)
(
)
/
(
...
)
(
)
/
(
)
(
)
/
(
)
(
)
(
...
)
(
)
(
)
(
2
2
1
1
2
1
n
n
n
A
P
A
B
P
A
P
A
B
P
A
P
A
B
P
B
P
A
B
P
A
B
P
A
B
P
B
P









 


 
 


n
i
n
i
i
i
i A
P
A
B
P
A
B
P
B
P
1 1
(9)
)
(
)
/
(
)
(
)
( 
(10)
)
(
)
/
(
)
(
)
/
(
)
/
(
1


 n
i
i
i
i
i
i
A
P
A
B
P
A
P
A
B
P
B
A
P
j
i A
B
A
B 
 and
32

Conditional Probability Cont’d…..
Example-1:
Solution:
)
/
(
1
)
/
(
)
/
(
)
/
(
1
obtain
we
,
)
(
by
sides
both
Dividing
)
(
)
/
(
)
(
)
/
(
)
(
)
(
)
(
)
(
B
A
P
B
A
P
B
A
P
B
A
P
B
P
B
P
B
A
P
B
P
B
A
P
B
P
B
A
P
B
A
P
B
P









 

)
/
(
1
)
/
(
that
Show B
A
P
B
A
P 

33

Example-2:
Let A and B be two events such that P(A)=x, P(B)=y and
P(B/A)=z. Find the following probabilities in terms of x, y and z.
Solution:
)
/
(
c.
)
(
b.
)
/
(
.
B
A
P
B
A
P
B
A
P
a

y
xz
B
P
B
A
P
B
P
B
P
B
A
P
B
A
P
xz
B
A
P
B
A
P
B
A
P
y
xz
B
P
B
A
P
B
A
P
xz
A
P
A
B
P
B
A
P














1
)
(
)
(
)
(
)
(
)
(
)
/
(
c.
1
)
(
1
)
(
)
(
b.
)
(
)
(
)
/
(
a.
)
(
)
/
(
)
(







34

Example-3:
A box contains two black and three white balls. Two balls are
selected at random from the box without replacement. Find the
probability that
a. both balls are black
b. the second ball is white
Solution:
First let us define the events as follows:
ball
black
a
is
selection
first
in the
outcome
the
:
1
B
35

ball
black
a
is
selection
second
in the
outcome
the
:
2
B
ball
white
a
is
selection
first
in the
outcome
the
:
1
W
ball
black
a
is
selection
second
in the
outcome
the
:
2
W
4
/
2
)
/
(
4
/
2
)
/
(
5
/
3
)
(
4
/
3
)
/
(
4
/
1
)
/
(
5
/
2
)
(
1
2
1
2
1
1
2
1
2
1






W
W
P
W
B
P
W
P
B
W
P
B
B
P
B
P
5
/
3
)
(
)
5
/
3
)(
4
/
2
(
)
5
/
2
)(
4
/
3
(
)
(
)
/
(
)
(
)
/
(
)
(
)
(
)
(
b.
10
/
1
)
(
)
5
/
2
)(
4
/
1
(
)
(
)
/
(
)
(
a.
2
1
1
2
1
1
2
1
2
1
2
2
2
1
1
1
2
2
1












W
P
W
P
W
W
P
B
P
B
W
P
W
W
P
B
W
P
W
P
B
B
P
B
P
B
B
P
B
B
P




36

Example-4:
Box A contains 100 bulbs of which 10% are defective. Box B
contains 200 bulbs of which 5% are defective. A bulb is
picked from a randomly selected box.
a. Find the probability that the bulb is defective
b. Assuming that the bulb is defective, find the probability
that it came from box A.
37

defective
is
Bulb
:
selected
is
Box
:
selected
is
Box
:
D
B
B
A
A
Solution:
First let us define the events as follows.

20
/
1
)
/
(
10
/
1
)
/
(
2
/
1
)
(
)
(




B
D
P
A
D
P
B
P
A
P
3
/
2
)
/
(
)
3
/
40
)(
20
/
1
(
40
/
3
20
/
1
)
(
)
(
)
/
(
)
/
(
b.
40
/
3
)
(
)
20
/
1
)(
20
/
1
(
)
2
/
1
)(
10
/
1
(
)
(
)
/
(
)
(
)
/
(
)
(
a.











D
A
P
D
P
A
P
A
D
P
D
A
P
D
P
B
P
B
D
P
A
P
A
D
P
D
P
38

Example-5:
One bag contains 4 white and 3 black balls and a second bag
contains 3 white and 5 black balls. One ball is drawn from the
first bag and placed in the second bag unseen and then one ball
is drawn from the second bag. What is the probability that it is
a black ball?
Solution:
First let us define the events as follows.
bag
first
the
from
drawn
is
ball
white
:
bag
first
the
from
drawn
is
ball
black
:
1
1
W
B
39

bag
second
the
from
drawn
is
ball
white
:
bag
second
the
from
drawn
is
ball
black
:
2
2
W
B
Then, we will have:
9
/
4
)
/
(
9
/
3
)
/
(
7
/
4
)
(
9
/
5
)
/
(
9
/
6
)
/
(
7
/
3
)
(
1
2
1
2
1
1
2
1
2
1






W
W
P
B
W
P
W
P
W
B
P
B
B
P
B
P
63
/
28
)
(
)
7
/
4
)(
9
/
5
(
)
7
/
3
)(
9
/
6
(
)
(
)
(
)
/
(
)
(
)
/
(
)
(
)
(
)
(
)
(
2
2
1
1
2
1
1
2
2
1
2
1
2
2










B
P
B
P
W
P
W
B
P
B
P
B
B
P
B
P
W
B
P
B
B
P
B
P 

40

Exercise:
1. For three events A, B and C, show that:
2. Box A contains 3 white and 2 red balls while another box B
contains 2 red and 5 white balls. A ball drawn at random from
one of the boxes turns out to be red. What is the probability that
it came from box A?
)
(
)
/
(
)]
/(
[
)
(
b.
)
/
(
)]
/(
[
]
/
)
[(
a.
C
P
C
B
P
C
B
A
P
C
B
A
P
C
B
P
C
B
A
P
C
B
A
P







41

3. In a certain assembly plant, three machine A, B and C make
30%, 45% and 25% of the products respectively. It is known
that 2%, 3% and 5% of the products made by each machine,
respectively, are defective. Suppose that a finished product is
randomly selected.
a. What is the probability that it is defective?
b. If the product is known to be defective, what is the
probability that it is made by machine A?
42

Independence of Events
 Two events A and B are said to be statistically independent if
and only if
 Similarly, three events A, B and C are said to be statistically
independent if and only if
 Generally, if A1, A2, …, An are a sequence of independent
events, then
)
(
)
(
)
( B
P
A
P
B
A
P 

)
(
)
(
)
(
)
( C
P
B
P
A
P
C
B
A
P 













 n
i
i
n
i
i A
P
A
P
1
1
)
(

43

Independence of Events Cont’d……
If A and B are independent, then we have
Example-1:
)
(
)
/
(
)
(
)
(
)
(
)
(
)
(
)
(
)
/
(
ii.
)
(
)
/
(
)
(
)
(
)
(
)
(
)
(
)
(
)
/
(
i.
B
P
A
B
P
B
P
A
P
B
P
A
P
A
P
B
A
P
A
B
P
A
P
B
A
P
A
P
B
P
B
P
A
P
B
P
B
A
P
B
A
P












t.
independen
also
are
and
that
show
then
t,
independen
are
and
If B
A
B
A
44

Solution:
Example-2:
The probability that a husband and a wife will be alive 90 years
from now are given by 0.8 and 0.9 respectively. Find the
probability that in 90 years
t.
independen
are
and
events,
t
independen
of
definition
By the
)
(
)
(
)]
(
1
)[
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
B
A
B
P
A
P
B
P
A
P
B
A
P
B
P
A
P
A
P
B
A
P
A
P
B
A
P
B
A
P
B
A
P
A
P

















alive
be
ll
neither wi
.
alive
be
will
one
least
at
.
alive
be
both will
.
b
c
a
45

Solution:
• First let us define the events as follows.
• Then we will have
• The two events can be considered as independent.
alive
be
will
Wife
:
alive
be
will
Husband
:
W
H
1
.
0
9
.
0
1
)
(
1
)
(
9
.
0
)
(
2
.
0
8
.
0
1
)
(
1
)
(
8
.
0
)
(














W
P
W
P
W
P
H
P
H
P
H
P
46

Solution:
0.98
one)
least
at
(
02
.
0
1
one)
least
at
(
)
neither
(
1
one)
least
at
(
.
02
.
0
)
(
)
(
)
1
.
0
)(
2
.
0
(
)
(
)
(
)
(
)
(
)
(
)
(
.
72
.
0
)
(
)
(
)
9
.
0
)(
8
.
0
(
)
(
)
(
)
(
)
(
)
(
)
(
.























P
P
P
P
c
B
H
P
neither
P
W
H
P
neither
P
W
P
H
P
W
H
P
neither
P
b
W
H
P
both
P
W
H
P
both
P
W
P
H
P
W
H
P
both
P
a






47

Assignment-I
1. Show that the probability that exactly one of the events A or B
occurs is given by:
2. If A and B are mutually exclusive events and P(A)=0.29,
P(B)=0.43, then find
3. A box contains 3 double-headed coins, 2 double-tailed coins
and 5 normal coins. A single coin is selected at random from
the box and flipped.
a. What is the probability that it shows a head?
b. Given that a head is shown, what is the probability that it is
the normal coin?
)
(
2
)
(
)
( B
A
P
B
P
A
P 


).
( B
A
P 
48

Assignment-I Cont’d……
4. The probability that a boy watches a certain television show is 0.4 and
the probability that a girl watches the show is 0.5. The probability that
a boy watches the show given that a girl does is 0.7. Find the
probability that
a. both of them watch the show
b. a girl watches the show given that her boy does
c. at least one of them watch the show
5. In a shooting test, the probability of hitting the target is 1/2 for A , 2/3
for B and 3/4 for C. If all of them fire at the target, find the probability
that
a. none of them hits the target
b. at most two of them hit the target
49

CHapter One -Basic_Probability_Theory.ppt

More Related Content

Similar to CHapter One -Basic_Probability_Theory.ppt (20)

Recently uploaded (20)

CHapter One -Basic_Probability_Theory.ppt