The Power of Base Rates: How Bayesian Thinking Beats Gut Feel

Your Gut is Lying to You

Many people play the lottery even though the odds are insane, but I bet they play because their gut tells them they are due. Ever hear “someone has to win it”? In Chile, they shout “el que la sigue, la consigue”, literally, “if you keep trying, you’ll get it.” It’s a great story. The truth is, the odds are absurd: matching 6 numbers in Canada’s 6/49 lotto is about 1 in 13,983,816. A $3 ticket only returns about $0.36 on average or a $2.64 loss each time. (If you want to see the math behind it, you can check it in the Risk Management section in the article below.)

So why did I buy one this week?

I know it’s a terrible bet. I’m literally writing this newsletter to warn you not to chase negative expected value bets. But deep down, my gut still whispered, “This is the one.” Maybe it’s fate setting me up for the ultimate irony of winning the jackpot right after I trash the odds. Or maybe it’s just because I used my lucky numbers, and hey, those have never let me down… except every other time.

The point is, I ignored the base rates. I knew the odds were awful, but I let a story and a feeling override the math. And that’s exactly what investors do all the time...skip the stats and bet big on what feels like a sure thing. That’s where Bayesian thinking comes in. It’s not about killing the story. It’s about starting with the base rate… and only adjusting if the evidence really earns it.

What Are Base Rates, and Why Do They Matter?

Base rates are simply the historical odds, the prior probabilities before adding any new details. Think of them as the statistical “starting line” or source of truth. For example, you may have heard “90% of restaurants fail,” but official data shows only about 17% fail in their first year. Either way, that number is your prior probability for a new eatery. In life and investing, there are priors everywhere: maybe ~10% of people are left-handed, or roughly half of marriages end in divorce. These are the base rates savvy thinkers start with.

Then comes updating. Bayesian logic says you take that prior and fold in new evidence. Bayesian thinking allows us to use all relevant prior information in making decisions. In practice, that means whenever a dramatic headline hits, ask yourself: “What did I know before?” If a crime-rate article says incidents doubled, your intuition might spike, but if the base rate was tiny to begin with, your actual risk is still low after updating.

The key is: do not toss out what the base-rate data already told you.

Bayesian Probability in Financial Modelling and Stock Market Prediction

Bayes’ Theorem provides a powerful framework for updating probabilities as new information comes in. This is especially useful in finance, where investors constantly revise their predictions based on evolving data like earnings reports, interest rate changes, or economic indicators.

Bayesian probability lets you start with an initial belief (a prior probability) and then adjust that belief when fresh evidence appears. In the context of stock markets, this means you can begin with a baseline expectation for an outcome (say, the chance a stock will rise) and refine that expectation as new news or data are observed.

The approach is valuable because markets are dynamic, new information arrives daily, and Bayes’ Theorem gives investors a systematic way to incorporate those updates into their decision-making in a conversational, intuitive manner (with just a light touch of math).

Understanding Bayes’ Theorem and Its Components

At its core, Bayes’ Theorem is a mathematical formula for determining conditional probabilities, the probability of an event given that some related condition holds. It provides a way to revise or update an existing prediction in light of new evidence. In finance, Bayes’ Theorem is often used to update risk evaluations or forecast probabilities of market events as new data becomes available. The theorem is named after 18th-century mathematician Thomas Bayes and underpins the field of Bayesian statistics.

In formula form, Bayes’ Theorem is written as:

P(A | B) = [P(A) x P(B|A)] / P(B)        

This equation might look abstract, so let’s break down each component in more relatable terms :

• Prior Probability (P(A)): This is your initial belief about event A before seeing new evidence. For example, prior could be the baseline chance that a stock’s price will increase based on historical data alone.
• Likelihood (P(B ∣ A)): This is the probability of observing evidence B assuming A is true. In investing, B could be some new information (like a positive earnings report), and this term reflects how likely that evidence would occur under scenario A (e.g., if the stock was truly destined to rise).
• Evidence Probability (P(B)): This is the overall probability of the new evidence B occurring, under all scenarios. In other words, how common is that news or signal in general.
• Posterior Probability (P(A ∣ B)): This is the updated probability of event A after incorporating the new evidence B. It’s what we get when we apply Bayes’ formula, essentially a revised forecast.

In words, Bayes’ Theorem says: Updated Probability = (Prior Probability × Likelihood of evidence) / Overall probability of evidence. This rule uniquely allows us to update our previous beliefs with new information in a quantitative way. If you know the prior odds of something (A) and you know how strongly some new information (B) is linked to that outcome, Bayes’ formula gives you the posterior odds of A after learning B.

For instance, if we let A = “Amazon’s stock price falls” and B = “the Dow Jones index fell today,” Bayes’ Theorem tells us:

P(Amazon falls | Dow down) = [P(Amazon falls) x P (Dow down | Amazon falls)] / P(Dow down)        

This matches the intuitive idea that the probability Amazon drops given a market drop equals the probability both Amazon and the Dow dropped, divided by the probability the Dow dropped. In other words, we’re adjusting the likelihood of Amazon’s decline by accounting for the new condition that the market is down.

Key takeaway: if you ever need to find “the probability of X given Y,” and you have some sense of X’s base-rate probability and how Y relates to X, Bayes’ rule is the tool to use . Financial analysts indeed use Bayes’ Theorem to forecast probabilities in the stock market as it enables them to refine predictions when an additional piece of information (a condition Y) is taken into account .

Applying Bayes’ Theorem to Investing and Stock Predictions

In the world of investing, conditions are always changing: interest rates fluctuate, companies release earnings reports, economic indicators surprise to the upside or downside, etc. Bayes’ Theorem offers a systematic way to incorporate these developments into your probability estimates of various outcomes. Rather than starting from scratch each time, you update your existing model. This is fundamentally a Bayesian approach: using prior beliefs plus new evidence to get updated beliefs.

Suppose you want to predict whether a stock index will go up or down tomorrow. Historically (looking at many past days of data), you might determine that the index increases on about 42.5% of days and decreases on about 57.5% of days (so your prior probability of a down day, A = “index down,” is 57.5%).

Now, imagine a new piece of information: interest rates have just increased today. Intuitively, you suspect that rising interest rates put downward pressure on stock prices. Bayes’ Theorem allows you to quantify that intuition by updating the probability of a down day given this news.

Using historical data, you find that interest rate increases (event B) occurred on 50% of the days in your sample, and in those instances, a large fraction of the time, the market fell. In fact, out of 2,000 trading days of data, interest rates rose on 1,000 days (P(B) = 0.50) and the index fell on 1,150 days (P(A) = 0.575). Crucially, 950 days saw both an interest rate rise and a stock index decline (this is the intersection of events A and B).

Given this information, we can apply Bayes’ formula:

P(Index down | Rates Up) = [(P(Index Down) x P(Rates Up | Index Down)] / P(Rates Up)        

Here P(Rates Up | Index Down) = 950 / 1150 = 0.826.

Plugging in the numbers:

P(Index Down | Rates Up) = [ 0.575 x 0.826 ] / 0.50 = 0.95

In other words, given that interest rates increased, the probability of the stock index decreasing jumps from the prior 57.5% to about 95% after updating with Bayes’ Theorem. We have revised our outlook dramatically upward for a market drop because the new evidence (rate hike) has a strong historical association with declines. The posterior probability (95%) is much higher than the prior (57.5%) because the evidence was strongly indicative of a downturn. This toy example shows how Bayes’ rule can use objective data to sharpen predictions instead of relying on guesswork.

For a simpler example, consider an individual stock and an earnings report. Imagine you initially believe a certain stock has a 60% chance of going up tomorrow (perhaps based on long-term historical trend), this is your prior, P(stock up) = 0.6.

Now, an earnings report comes out today. Let’s say it’s positive news, which in the past has been correlated with stock price increases. Suppose historically, when such positive news hits, the stock ended up rising 70% of the time (and conversely, if the news were bad, maybe the stock only rose 30% of the time). Using Bayes’ Theorem, you update the 60% prior with this new evidence. Without going into all the math, the result might be an updated probability of, say, ~78% that the stock will rise given the favourable report.

The exact figure comes from applying the formula with the “likelihood” of a rise given good news and the overall probability of such good news, but qualitatively, the takeaway is that your confidence in the stock’s rise should increase when you incorporate the new info. If the news had been bad, the formula would likewise lower the probability of a price increase.

The Importance of Base Rates (Prior Probabilities) in Decision Making

One of the classic insights Bayes’ Theorem provides is the importance of base rates (the prior probabilities) in our decision making. A common pitfall known as the base rate fallacy is when people ignore these base rates and focus only on the new evidence or specific scenario at hand.

This tendency was highlighted by psychologists Amos Tversky and Daniel Kahneman: people often get so caught up in case-specific information that they overlook the broader statistical context. Bayesian thinking guards against this by mathematically ensuring the base rate (prior) is factored into the updated probability.

A famous illustration is the story of “John,” who wears gothic clothing and listens to death metal. Many would instinctively guess John is more likely to be a Satanist than a Christian based on that vivid description.

However, the base rates of those groups (Christians vastly outnumber Satanists in the world) mean John is statistically far more likely to be Christian despite the niche appearance.

The base rate fallacy is our intuitive failure to account for that huge prior difference. Bayes’ Theorem, on the other hand, would take John’s appearance as evidence that updates the probability, but it would also heavily weight the prior odds (2 billion vs very few), correctly resulting in a much higher probability that John is Christian in the end. In short, the new evidence can tilt the odds, but the prior baseline remains a crucial part of the calculation.

Translating this to investing: an investor might get excited about a hot new tech startup and overestimate its chance of success because of some flashy news or storytelling, while forgetting that, say, the base rate of success for startups in that sector is only 1 out of 10. A Bayesian investor would acknowledge that the positive news does increase the probability of the startup’s success somewhat, but would also remind themselves of the low base success rate, tempering the final estimate.

By quantifying this, Bayes’ theorem helps avoid overconfidence in flashy but low-probability bets. There’s an old saying: “It’s more important to invest in the right sector than the right stock.” In Bayesian terms, that’s because a strong sector provides a favourable base rate for success (many companies in that sector do well on average), effectively a high prior probability. Choosing a booming sector (high base-rate of returns) and investing in a basket of its stocks can sometimes yield better odds than zeroing in on one company in a struggling sector, no matter how good that single company’s story sounds. This insight reflects Bayesian thinking: focus on prior probabilities (sector-wide odds) before getting carried away with specific “evidence” about an individual stock.

That’s why you’ll notice my portfolio tends to lean into industries with a high base rate of success (like semiconductors) and steers clear of those with structurally weak odds, like airlines.

I’d rather bet on a rising tide than hope one struggling boat stays afloat. Base-rate thinking helps me focus on where the long-term odds are already stacked in my favour.

By properly weighting base rates, Bayes’ theorem can improve decision-making under uncertainty. It forces us to formally account for what we already know (historical frequencies, long-term trends) when interpreting new clues. This can prevent common biases like overreacting to recent news or putting too much faith in one expert prediction. Instead, each new data point nudges the prior belief up or down in a reasoned way.

Real-World Uses of Bayesian Methods in Finance

Bayesian methods have numerous practical applications in finance and investing, thanks to their ability to continuously update forecasts.

For example,

• Risk Assessment and Lending: Banks and lenders use Bayesian approaches to continually update the estimated risk of a borrower defaulting. For instance, as a borrower’s credit history grows or as economic conditions change, Bayes’ Theorem can revise the probability of default, leading to more accurate pricing of loans. Rather than one static credit score, the risk evaluation becomes a living estimate that improves with more information.
• Stock Price Prediction and Trading Strategies: As we discussed, traders can use Bayesian models to forecast stock movements by blending prior market behaviour with current news. If an analyst has a prior model for a stock’s price behaviour, each new earnings report or economic indicator can feed into a Bayesian update. This is the principle behind some algorithmic trading strategies that incorporate news feeds: the algorithm updates probabilities of market scenarios in real-time as new data arrives, aiming to achieve a level of “Bayesian precision” in its predictions. Over time, these updates can improve the accuracy of trend predictions and help traders make data-informed decisions (buy, sell, or hold) based on the latest evidence without throwing away prior knowledge. This is not my investment approach but it is the approach of some 😊
• Portfolio Management and Asset Allocation: Bayesian inference isn’t limited to single-event predictions; it can also enhance portfolio-level decisions. Portfolio managers can use Bayesian models to update expected returns and volatility for various assets as new information comes in. For example, if new economic data suggests higher inflation, a Bayesian framework might increase the expected return (and risk) of commodities in the portfolio while adjusting expectations for bonds. Bayesian portfolio optimization means the asset allocation can dynamically shift based on the latest data, rather than sticking to a fixed assumption. This adaptability can help in reducing risk and boosting returns over time by responding promptly to real-world developments .
• Economic and Market Forecasting: Beyond individual stocks, Bayes’ Theorem is used to update forecasts of broader economic metrics (GDP growth, interest rate changes) or market regimes. Analysts often start with priors (say, an X% chance the Federal Reserve will raise rates next quarter) based on historical patterns and current baseline conditions. Then they update those probabilities when new signals appear (perhaps a sudden jump in inflation or a change in Fed officials’ tone) to get a revised probability of the rate hike scenario. This approach provides a disciplined way to handle evolving information, where each new data point refines the probability of various outcomes rather than flipping predictions on a whim.

It’s worth noting that Bayesian analysis has become even more practical with modern computing. Techniques like Bayesian networks (graphical models of probabilistic relationships) and Markov Chain Monte Carlo (MCMC) simulations allow investors to deal with complex, multi-factor Bayesian models.

For instance, a Bayesian network might model how interest rates, employment data, and corporate earnings collectively influence stock returns, updating each node of the network as new data comes in. MCMC methods help in estimating probabilities when direct calculation is too complex, by simulating many scenarios and converging on a posterior distribution. These advanced tools extend Bayes’ basic idea to very sophisticated financial models, enabling adaptive strategies that learn from data in real time.

Where Investors Go Wrong Without Base Rates

Investors often fall into shiny traps by ignoring base rates:

• Unicorn delusions: Believing every startup will disrupt something. In reality, up to 90% of startups fail, so most “next big things” never are. Ignoring that prior means overpaying for hype.
• Margin mania: Thinking today’s ultra-high profit margins will last forever. Economics research shows margins tend to revert toward the long-term norm over time. If a company has a 50% margin now, base rates suggest competitors and normal cycles will push it down eventually.
• Turnaround trap: Assuming every management reboot fixes the business. Only about 1 in 4 corporate turnarounds succeed long-term. That means 75% of “comeback” stories end in disappointment.

In short: skip base rates and you’ll end up paying a premium for a feel-good story instead of reasonable odds.

But Wait: Isn’t Gut Feeling Valuable?

Sometimes, yes, but only when it’s earned. A “gut” hunch is your brain’s pattern recognition at work. Experts can thin-slice, seeing signals in a sliver of data. Gladwell notes that “experts… can make highly accurate judgments based on minimal information”. Think of a firefighter instantly reading a fire scene or a coach seeing a play unfold. Their intuition works because their brains have logged so much experience.

Psychological research shows our unconscious mind can integrate tons of details. In one set of experiments, people who “sleep on” a complex decision often make better choices than those who overthink every point. That’s your subconscious crunching data you didn’t even consciously process.

But (and this is big) most people confuse gut with guesswork. If you haven’t fed your instinct with real experience and data, that feeling is just noise.

Remember: the gut isn’t magic, it’s memory + instinct. If it hasn’t “eaten” good base-rate knowledge and reps, it’s just indigestion.

The Bayesian Upgrade: Combining Gut + Base Rates

I blend priors with intuition. Here’s a simple framework:

1. Anchor on the base rate. Start with the historical odds (the prior) before you get excited.
2. Adjust cautiously. If your experience or analysis suggests a case is special, explicitly quantify how. Why should the odds be better or worse than the base rate?
3. Demand evidence. Ask: “What data justifies my adjustment?” If there’s no solid proof, don’t stray too far from the prior.

In practice: don’t just whisper “It feels right.” Whenever you form a hunch, ask “What are the relevant priors?”. Only after you clearly identify a reason (like a rare contract, a unique edge, etc.) should you shift off the base rate.

Final Thought: Boring Wins

It may sound unglamorous, but Bayesian thinking is all about stacking the odds in your favour. Exciting stories sell, but steady probabilities compound your returns. Investing isn’t a movie plot, it’s about avoiding dumb traps and staying rational. As the saying goes: “Your gut can be right – just make sure it ate some base rates first.”

Next article drops on Wednesday.

Unless, of course, my lucky numbers finally hit. In that case… Bayesian who?

For more deep dives where data-driven analysis meets stock picks, check out Beating the Tide, where base rates + intuition = alpha.