Bayesian Analysis Fundamentals with Examples

Bayes enables
updating beliefs
by (fancy) counting

Bayes’ Theorem - What is it good for?

Some Zynga-ish examples
Bayesian A/B testing (T2 subsidiary)
Mixed Media Model (Cathy)
Creative Health Monitoring (Ivy)
Growth Scorecard

What to expect from this workshop
What is it?
Why is it true?
When is it useful?
How to use it?
Specifically how to use it in my models
Levels of Understanding

Who is Steve?
“Steve is very shy and withdrawn,
invariably helpful but with very little
interest in people or in the world of
reality. A meek and tidy soul, he has a
need for order and structure, and a
passion for detail.” Is Steve more likely
to be a librarian or a farmer?
10% 90%

Kahneman and Tversky
reality. A meek and tidy soul, he has
a need for order and structure, and a
passion for detail.” Is Steve more
likely to be a librarian or a farmer?
10% 90%

Spoiler Alert
We are going to describe this in pictures
20
1

200
10
reality. A meek and tidy soul, he has
a need for order and structure, and a
passion for detail.” Is Steve more
likely to be a librarian or a farmer?

4
40%
10%
20
P(librarian | description) =

Heart of Bayes Theorem
All
Possibilities
All
Possibilities
fitting
evidence
P(Librarian
given the
evidence)

For each
possibility
Count the
number of
ways evidence
can happen
Make this
ratio:
P(Librarian
given the
evidence)

When to use Bayes Rule
We have a
hypothesis
We observe some
evidence
“Steve is very shy and
withdrawn, invariably helpful
but with very little interest in
people or in the world of
reality. A meek and tidy soul,
he has a need for order and
structure, and a passion for
detail.”
We want
P(H|E)
P(hypothesis given
evidence)

All
Possibilities
All
Possibilities
fitting
evidence
P(Librarian
given the
description)
How can we write
this more
mathematically?

Goal: P(H|E)
P(hypothesis given
evidence)
P(Librarian given
description)

Goal: P(H|E)
Prior → P(H) = 1/21

Goal: P(H|E)
Prior → P(H) = 1/21
P(E|H) = 0.4

Goal: P(H|E)
Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood

Goal: P(H|E)
Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
P(H|E)

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
P(H|E) =

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) =

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) = =
(# )
210

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) = =
(# ) P(H)
10

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) = =
(# ) P(H) P(E|H)
4

P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) = =
(# ) P(H) P(E|H)
(# ) P(H) P(E|H) + (# ) P(￢H)P(E|￢H)
4
4 20
P(￢H) = 20/21

Prior → P(H) = 1/21
P(E|H) = 0.4
Likelihood
P(E|￢H) = 0.1
+
P(H|E) =
P(H) P(E|H) Bayes
Theorem
P(E)
Posterior

P(H|E) =
P(H) P(E|H) Bayes
Theorem
P(E)

P(H|E) =
P(H) P(E|H) Bayes
Theorem
P(E)
1
1

P(H|E) =
P(H) P(E|H) Bayes
Theorem
P(E)
1
1
Final Note About Steve

Bayesian Data Analysis
in 3 easy steps!
1. For each possible explanation
of the data
2. Count all the ways data can
happen
3. Explanations with more ways
to produce the data are more
plausible (make a ratio)

Bayesian Updating
in 3 easy steps!
1. Make a model for how observations can happen. Do it
again for each possible explanation
2. Count all the ways data can happen for each
explanation
3. The relative values of step 2 give the relative
plausibility

What
proportion of
the earth is
covered by
water?
demo

1. For each possible
explanation of the
data
2. Count all the ways
data can happen
3. Make a ratio
Get more Evidence and
Update

Bayes enables
updating beliefs
using MCMC

When to use Bayesian inference?
● Data is limited
○ Updating Bayesian estimates converge with modest data support
○ In large sample size limit, Bayesian and Frequentist converge
● Uncertainty is important
○ Bayesian estimates are distributions
○ Probabilities are conducive to confidence metrics
● Quantifiable prior beliefs
○ We know something about the world ahead of time
○ Easy to update if what we know changes

Parachute
Manufacturing
Bayesian Workshop

Take-Two: Parachute Division
Strauss just announced that Take-Two is going to pivot it’s Zynga
Mobile division to solely manufacturing parachute toys, called ‘Chutes.
Our new mission is to “Connect the world through ‘Chutes”.
Manufacturing has already started, and we have the first batch fresh
off the assembly line. Uh oh! In the frenzy of the re-org, we forgot to
characterize the aerodynamic performance of this toy—namely, it’s
drag coefficient—which is a requirement by the World Aerial Toy
Association.
Legal said we are vulnerable to huge fines if we can’t provide the drag
coefficient in our Terms of Service by the time we go live with our new
toy. We have 1 hour to solve this problem!
Our goal, is to devise an experiment that uses a Bayesian approach to
quickly and confidently estimate the ‘Chute’s drag coefficient.

Aerodynamics Overview
Luckily, Take-Two has a resident Rocket
Scientist on retainer. Dr. Ryan has offered
to give some background aerodynamics
information to get us started.
First, let’s draw a Free Body Diagram.
Weight
(W)
Drag (D)

Our manufacturer provided
us the weight measurement.
Equation for Drag force:
Density of air
Reference area
Velocity
Drag coefficient
What is the stable,
unaccelerated state of a ‘Chute?
When the Drag force balances
Weight, we are gliding at a
terminal velocity.
Let’s assume this happens fairly
fast after initial drop.
Weight
(W)
Drag (D)

Once a ‘Chute is at a terminal
velocity, Drag = Weight. We can
solve for V.
Great, but how can we find Cd?
What if we try to measure V
through an experiment?
Weight
(W)
Drag (D)

Drag Coefficient Experiment
Proposed experiment:
Drop for a known height and record the
glide time. We know:
Frequentist approach:
Measure a bunch of distance
and time. Calculate Cd, and
report the average value.
The only
unknown is Cd

Bayesian Approach
Our experiment generates data. Our unknown
random variable 𝛉 is Cd.
Likelihood:
For a given value of Cd, what’s the probability
of generating our data? We can use our physics
model!
Prior:
Our best guess of the probability distribution of
a ‘Chute’s drag coefficient.
Posterior:
Updated guess of Cd, updated given the set of
data we collected.

Bayesian Approach
Likelihood:
Prior:
Expert consultant:
Posterior:
PyMC3 package will solve for this, and will give
us posterior distributions for Cd.
*Our experimental error recording
time t is normally distributed

Challenge
Goal: Get Strauss and Legal a Bayesian estimate of Cd!
Split up into groups of two, and conduct your experiment:
1. Clone this Databricks notebook: https://dbc-019b8f42-
900e.cloud.databricks.com/?#notebook/4116061
2. Drop the ‘Chute at least 10 times from a known height and record it’s freefall time.
Enter your data in the Databricks notebook.
3. Notice your prior distribution for Cd, given to you by your Expert Consultant
4. Run PyMC3 to calculate the posterior estimate of Cd. Record the distribution’s mean and standard
deviation.
5. Plot your posterior estimates of Cd after:
a. No data, plot the prior
b. 1 data point
c. 5 data points
d. 10 data points
We will reconvene and discuss our results after 45 minutes!

Reference Material
● Bayesian parameter estimation examples:
https://epubs.siam.org/doi/pdf/10.1137/100788604
https://cimec.org.ar/ojs/index.php/mc/article/download/5564/5542
https://arxiv.org/abs/2104.08621
● A great Bayes intro w/ Regression examples:
https://bayesball.github.io/BOOK/
https://www.bayesrulesbook.com/
● Bayesian A/B testing POC @ Zynga:
https://github-ca.corp.zynga.com/kryan/BayesRozesPOC

Bayes is
just
counting
computing
the
posterior

MCMC
You can evaluate the function, but you
cannot draw sample from the function

When to use Bayesian inference?
● When data is lacking
○ Frequentist: large error bars, poor estimates
○ By making more assumptions, we get to leverage more information
○ In large sample size limit, Bayesian and Frequentist approaches
converge
● When you desire levels of belief over True/False
statements
○ Bayesian approach still has distribution for parameter, even for large
sample size
○ Bayesian interpretation of probability is more intuitive to people

Bayes’ Theorem
likelihood evidence
posterior
Observations, data, features
Outcome, label

Bayesian POV
● Experiment → prior notion + data = new (posterior) notion
● Unknown parameters → associated probabilities interpreted as
“belief” in truth
● Probabilities → how well a proposition is supported by the data
provided as evidence for it

Bayesian Analysis Fundamentals with Examples

More Related Content

Similar to Bayesian Analysis Fundamentals with Examples (20)

More from Tushar Tank (13)

Recently uploaded (20)

Bayesian Analysis Fundamentals with Examples