Probability Theory and statistics tome 1
- 1. Probability Theory Probability and Statistics for Data Science CSE594 - Spring 2016
- 3. What is Probability? Examples ● outcome of flipping a coin (seminal example) ● amount of snowfall ● mentioning a word ● mentioning a word “a lot” 3
- 4. What is Probability? The chance that something will happen. Given infinite observations of an event, the proportion of observations where a given outcome happens. Strength of belief that something is true. “Mathematical language for quantifying uncertainty” - Wasserman 4
- 5. Probability (review) Ω : Sample Space, set of all outcomes of a random experiment A : Event (A ⊆ Ω), collection of possible outcomes of an experiment P(A): Probability of event A, P is a function: events→ℝ 5
- 6. Probability (review) Ω : Sample Space, set of all outcomes of a random experiment A : Event (A ⊆ Ω), collection of possible outcomes of an experiment P(A): Probability of event A, P is a function: events→ℝ ● P(Ω) = 1 ● P(A) ≥ 0 , for all A ● If A1 , A2 , … are disjoint events then: 6
- 7. Probability (review) Ω : Sample Space, set of all outcomes of a random experiment A : Event (A ⊆ Ω), collection of possible outcomes of an experiment P(A): Probability of event A, P is a function: events→ℝ P is a probability measure, if and only if ● P(Ω) = 1 ● P(A) ≥ 0 , for all A ● If A1 , A2 , … are disjoint events then: 7
- 8. Probability Examples ● outcome of flipping a coin (seminal example) ● amount of snowfall ● mentioning a word ● mentioning a word “a lot” 8
- 9. Probability (review) Some Properties: If B ⊆ A then P(A) ≥ P(B) P(A ⋃ B) ≤ P(A) + P(B) P(A ⋂ B) ≤ min(P(A), P(B)) P(¬A) = P(Ω / A) = 1 - P(A) / is set difference P(A ⋂ B) will be notated as P(A, B) 9
- 10. Probability (Review) Independence Two Events: A and B Does knowing something about A tell us whether B happens (and vice versa)? 10
- 11. Probability (Review) Independence Two Events: A and B Does knowing something about A tell us whether B happens (and vice versa)? ● A: first flip of a fair coin; B: second flip of the same fair coin ● A: mention or not of the word “happy” B: mention or not of the word “birthday” 11
- 12. Probability (Review) Independence Two Events: A and B Does knowing something about A tell us whether B happens (and vice versa)? ● A: first flip of a fair coin; B: second flip of the same fair coin ● A: mention or not of the word “happy” B: mention or not of the word “birthday” Two events, A and B, are independent iff P(A, B) = P(A)P(B) 12
- 13. Probability (Review) Conditional Probability P(A, B) P(A|B) = ------------- P(B) 13
- 14. Probability (Review) Conditional Probability P(A, B) P(A|B) = ------------- P(B) 14 H: mention “happy” in message, m B: mention “birthday” in message, m P(H) = .01 P(B) =.001 P(H, B) = .0005 P(H|B) = ??
- 15. Probability (Review) Conditional Probability P(A, B) P(A|B) = ------------- P(B) 15 H: mention “happy” in message, m B: mention “birthday” in message, m P(H) = .01 P(B) =.001 P(H, B) = .0005 P(H|B) = .50 H1: first flip of a fair coin is heads H2: second flip of the same coin is heads P(H2) = 0.5 P(H1) = 0.5 P(H2, H1) = 0.25 P(H2|H1) = 0.5
- 16. Probability (Review) Conditional Probability P(A, B) P(A|B) = ------------- P(B) Two events, A and B, are independent iff P(A, B) = P(A)P(B) P(A, B) = P(A)P(B) iff P(A|B) = P(A) 16 H1: first flip of a fair coin is heads H2: second flip of the same coin is heads P(H2) = 0.5 P(H1) = 0.5 P(H2, H1) = 0.25 P(H2|H1) = 0.5
- 17. Probability (Review) Conditional Probability P(A, B) P(A|B) = ------------- P(B) Two events, A and B, are independent iff P(A, B) = P(A)P(B) P(A, B) = P(A)P(B) iff P(A|B) = P(A) Interpretation of Independence: Observing B has no effect on probability of A. 17 H1: first flip of a fair coin is heads H2: second flip of the same coin is heads P(H2) = 0.5 P(H1) = 0.5 P(H2, H1) = 0.25 P(H2|H1) = 0.5
- 19. Why Probability? A formality to make sense of the world. ● To quantify uncertainty Should we believe something or not? Is it a meaningful difference? ● To be able to generalize from one situation or point in time to another. Can we rely on some information? What is the chance Y happens? ● To organize data into meaningful groups or “dimensions” Where does X belong? What words are similar to X? 19
- 20. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. 20
- 21. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} 21
- 22. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 22
- 23. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 23
- 24. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 What is the probability that we end up with k = 4 tails? P(X(ω) = k) where ω ∊ Ω 24
- 25. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 What is the probability that we end up with k = 4 tails? P(X = k) := P( {ω : X(ω) = k} ) where ω ∊ Ω 25
- 26. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 What is the probability that we end up with k = 4 tails? P(X = k) := P( {ω : X(ω) = k} ) where ω ∊ Ω X(ω) = 4 for 5 out of 32 sets in Ω. Thus, assuming a fair coin, P(X = 4) = 5/32 26
- 27. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 What is the probability that we end up with k = 4 tails? P(X = k) := P( {ω : X(ω) = k} ) where ω ∊ Ω X(ω) = 4 for 5 out of 32 sets in Ω. Thus, assuming a fair coin, P(X = 4) = 5/32 (Not a variable, but a function that we end up notating a lot like a variable) 27
- 28. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = 5 coin tosses = {<HHHHH>, <HHHHT>, <HHHTH>, <HHHTH>…} We may just care about how many tails? Thus, X(<HHHHH>) = 0 X(<HHHTH>) = 1 X(<TTTHT>) = 4 X(<HTTTT>) = 4 X only has 6 possible values: 0, 1, 2, 3, 4, 5 What is the probability that we end up with k = 4 tails? P(X = k) := P( {ω : X(ω) = k} ) where ω ∊ Ω X(ω) = 4 for 5 out of 32 sets in Ω. Thus, assuming a fair coin, P(X = 4) = 5/32 (Not a variable, but a function that we end up notating a lot like a variable) X is a discrete random variable if it takes only a countable number of values. 28
- 29. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. X is a discrete random variable if it takes only a countable number of values. X is a continuous random variable if it can take on an infinite number of values between any two given values. 29
- 30. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = inches of snowfall = [0, ∞) ⊆ ℝ 30 X is a continuous random variable if it can take on an infinite number of values between any two given values.
- 31. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = inches of snowfall = [0, ∞) ⊆ ℝ X amount of inches in a snowstorm X(ω) = ω 31 X is a continuous random variable if it can take on an infinite number of values between any two given values.
- 32. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = inches of snowfall = [0, ∞) ⊆ ℝ X amount of inches in a snowstorm X(ω) = ω What is the probability we receive (at least) a inches? P(X ≥ a) := P( {ω : X(ω) ≥ a} ) What is the probability we receive between a and b inches? P(a ≤ X ≤ b) := P( {ω : a ≤ X(ω) ≥ b} ) 32 X is a continuous random variable if it can take on an infinite number of values between any two given values.
- 33. Random Variables X: A mapping from Ω to ℝ that describes the question we care about in practice. Example: Ω = inches of snowfall = [0, ∞) ⊆ ℝ X amount of inches in a snowstorm X(ω) = ω What is the probability we receive (at least) a inches? P(X ≥ a) := P( {ω : X(ω) ≥ a} ) What is the probability we receive between a and b inches? P(a ≤ X ≤ b) := P( {ω : a ≤ X(ω) ≥ b} ) P(X = i) := 0, for all i ∊ Ω (probability of receiving exactly i inches of snowfall is zero) 33 X is a continuous random variable if it can take on an infinite number of values between any two given values.
- 34. Probability Review ● what constitutes a probability measure? ● independence ● conditional probability ● random variables ○ discrete ○ continuous 34