6 Base Rates and Reference Classes
Let’s start by considering the following question:
What is the probability that Joe Biden is President of the United States on Nov. 1st, 2024?
Brainstorm a few factors / historical events that have caused presidents to leave office before the end of their elected term. What is the base rate of these events? In other words, what is the rate of occurrence of these events?
Base rates can be powerful as they allow us to draw analogies with related cases even when we don’t have directly relevant data or a trend line to extrapolate. In that sense they are like zeroth-order forecasting, but can work even when you don’t have a clear time series or other trend to base a forecast on.
6.1 Decomposing the Problem
Let’s only take these key considerations in why Biden wouldn’t complete his term:
Death by natural causes
Death by assassination
Impeachment / resignation
The presidency no longer exists (due to a coup, invasion, etc.)
What did we gain from applying this decomposition? The main thing is that it allows us to incorporate age when predicting death from natural causes, which feels correct to me. On the other hand, it left us with a very small sample size for assassinations, and in general left us to make a lot of arbitrary choices that might let personal biases creep in. That being said, the number is in the same ballpark but a bit lower than the 92% answer from the simplest method, which feels right given Biden’s age.
6.2 Other Examples
There are many cases where base rates are a useful tool. For instance, maybe I want to understand the probability that I get COVID-19 in the next month (perhaps as a function of what activities I do).
Brainstorming exercise. What base rates would you use for the above question? What factors are most important to take into account?
Here are some other questions where base rates provide valuable information:
- What will be my GPA after my first term in college?
- How much Series A funding will a specific startup get?
- Will someone break Usain Bolt’s 100-meter dash 9.58s world record by the end of 2024?
- Will any new country join NATO in 2023?
*Brainstorming exercise. What are other areas where base rates are helpful?
6.3 Dangers of Base Rates
As I hinted above, the flexibility of base rate forecasting also carries risk. If we make too many arbitrary choices when defining a base rate, we can succumb to our own cognitive biases. For instance, consider the following article that gave Trump a 3% chance of winning the 2016 election. Its reasoning invokes base rates, defining a reference class of “Candidates that are good with the media and give them something to write about but, let’s be real, could never be president”. They said Trump was in this reference class and no such candidate had ever won, so their base rate was 0%. (They then adjusted it up to 3% based on Trump’s strong polling performance.) If you find yourself doing something like this, watch out.
The best ways to avoid failures like the Trump prediction are to look for the simplest reference classes you can find (only adjusting for obviously important and objective factors like age), or to average over lots of ways of constructing your reference class so that no single set of choices dominates the forecast.