You might have already heard about biases and how they affect our decision-making process, oftentimes leading to suboptimal results. These play a major role in the financial markets, and have thus been study in great detail.
Amongst these, survivorship bias stands out as one of the most dangerous and often difficult to identify ones.
Even agents with relatively passive strategies that consist in rebalancing a portfolio every now and then are exposed might be exposed to survivorship bias, which in turn leads to overly optimistic backtesting results. This in turn probably will lead to disappointing future returns.
In this article, we will discuss survivorship bias and why it is important for all types of traders to be aware of its existence. So, without any further ado, lets get started!
What is survivorship bias?
Survival bias is the logical fallacy of focusing an analysis on people, things or in a generic sense, samples that have been through the selection process or a filter, thus ignoring those cases that have been filtered out. This type of bias is oftentimes caused due to the lack of knowledge and expertise of analysts about the phenomena being studied. As is the case with any bias, survivorship bias can lead to formulating wrong conclusions.
Survivorship bias has been broadly documented in various fields of study, ranging from economics to evolutionary biology. So, we can define survivorship bias as the logical error of sampling based on a pool of “survivors” of a process and omitting cases who did not survive the sampling process due to some ignored effect.
When analyzing data, it’s important to ask yourself if your sampling process is prone to filtering out relevant data. The most famous example of survivorship bias affecting a sampling process dates back to WW2 when a team of scientists was tasked with deciding which parts of combat airplanes should be reinforced with additional protection. The planes that returned from battle had bullet holes everywhere except the engine and cockpit. The team decided (correctly) that it was better to fit armor where there were no bullet holes because planes because were the fatal places. Planes with holes in those places did not make it.
This type of bias also applies to the financial market in times of crisis and/or other financial events, and not accounting for this bias can lead to extreme optimism because “failures” are ignored.
A clear example of optimism caused by survivorship bias in the financial industry is the common mistake of creating a portfolio consisting of a handful of the SP500 constituents and calculating the historical profitability of the portfolio. We are, by definition choosing only companies that did not go bankrupt in the past. Moreover, we are also selecting historically successful companies insofar as they are currently constituents of the SP500. In order to be rigorous, we should have chosen the companies from a pool of available companies at the start of the backtesting period. You can further read about common backtesting mistakes in this article.
Survivorship Bias in Financial Markets
We like success stories about traders/investors, and we “forget” to look at the overall odds of being successful. Failures, except for incredible failures, tend to be ignored or forgotten.
When hedge funds and mutual funds develop marketing materials for their products, they will often include a benchmark (S&P 500, MSCI Hedge Fund) and report their performance compared to peers (funds with a similar strategy or risk profile). They often use a universe of funds that excludes dead or merged funds, making their returns seem safer. This may result in a discrepancy of up to 1.6% additional return yield (Alpha).
Indeed, survivorship bias influences much of how we perceive the world and can have a very real impact, both positive and negative.
Another example of survivorship bias is associated with companies that are successful enough to be recently included in an index. Due to the fact that they recently reached the criteria to be included, it is logical for their recent performance to have been unusually good.
This is described as “the pre-rating inclusion error” or type-2 survivor bias. This could introduce abnormally high returns if you were to include a company that was added to the index today versus a company that has been in the index for some time.
Another example is related to IPOs. If your strategy consists in buying companies on their IPO date, you should consider the roughly 230 companies that IPO each year, and not only the ones that you can remember. The fact that you know their name is a clear symptom of survivorship bias. In 2010, both Tesla and Molycorp went public. It is safe to say that their performance differed. Molycorp filed for bankruptcy in 2015, whereas Tesla seems to be doing just fine.
Common causes of Survivorship Bias
Thinking that a company is going to repeat past results, and grow at the same rate as it has been doing, is a fallacy that many investors fall for. Or to think that since the company traded at $20 a share and is now trading at $5, it’s a market bargain. Stocks don’t know record highs, and companies have no inherent physical property that forces them to repeat past performance. This is one of the main causes of flawed backtests.
Backtesting is the process of simulating a trading strategy with the help of historical data. This method, a common technique in the financial industry, allows for evaluating the profitability of a given strategy by iterating and discarding unprofitable trading ideas.
Most stock market indices are weighted averages of market capitalization. However, averages are, in most cases, gross oversimplifications and generalizations that are not very relevant. These indices can thus be oftentimes misleading because they rarely reflect the performance of the “average stock.”
Take Foot Locker as an example. It was delisted from the S&P500 on August 9, 2019. Its price was down roughly -30% in the last five years, while the S&P500 went up by a whopping 51%.
If the current constituents of the S&P 500 are your benchmark, then you won’t have Foot Locker in your data set. As a consequence, your trading strategy has no chance of selecting this underperforming asset. Foot Locker’s negative performance was reflected in the benchmark S&P500 but would be missing if you retroactively calculated its performance based on the current list of companies that comprise it. This logical error will falsely lead you to believe that your trading strategy is better than it is.
As a corollary, one of the most common mistakes of backtesting is to do it only for companies that did not go bankrupt. Carrying out a backtest on companies that did not go bankrupt does not allow us to have the idea that there is a probability that they will go bankrupt.
As we mentioned before, imagine having invested in Lehman Brothers before it went bankrupt in 2008. A 20% drop was not a buying opportunity.
Another case was that of the insurer AIG, which also suffered a major blow in the 2008 crisis, where it had to be bailed out by the US government.
AIG once traded at $1,700 per share. A 50% drop in the stock price brought it to $850 per share. You might wonder what the current price is: $48 per share.
Frequently Asked Questions (FAQ)
How to get Survivorship Bias-Free Data
Obtaining data that is free of survivorship date is surprisingly difficult since most providers of historical constituent data require a paid subscription or at least a one-time purchase. The closest and most reliable free resource I found is here.
With this in mind, the S&P500 index can be approximated by its ETF, the iShares Core S&P 500 ETF (IVV), which discloses its holdings each month and claims to invest at least 90% of its assets in index securities.
To import the price base, you can use two web pages: Yahoo Finance and WIKI Prices Dataset (I’ll copy the link in case you want to link it at the end).
What percentage of data is missing due to survivorship bias?
To be included in the SP500 index, a company needs a market capitalization of at least $13.1 billion, in addition to having positive earnings. It also has to be highly liquid and be based in the US. Last but not least, the majority of its shares must be in public hands.
One way to investigate changes in the index over time is to review changes in sector weights. The S&P is currently heavily weighted towards the information technology sector, which accounts for approximately 28% of the index. Its relative weight has grown significantly over the past five years, from just over 21% in 2016.
The biggest change during that time span, however, is in Communication Services, which jumped from 2% of the index to nearly 11% today. This increase was primarily driven by sector reclassifications that added Meta, Alphabet, Netflix, and Disney to the sector. At the same time, both the consumer staples and energy sectors have seen significant weight reductions compared to five years ago.
Contrary to popular belief, the number of publicly traded US companies decreased during the 21st century, going from roughly 5.500 companies in the year 2000 to only 4.000 in 2020.
Last but not least, the following list comprises the companies that went filed for bankruptcy in 2020 alone:
Also, looking at the weights between 2008 and 2022, you can see 2022s massive “tech boom” and a decrease in energy companies. Also, the fall in soft drinks, snacks, and tobacco can be seen. Incredibly, ten companies from September 2008 are still the top 20 companies in the index today. What follows is a list of the top 20 companies (measured in terms of market capitalization) both for 2008 and 2022.
As can be seen, survivorship bias is not only an important factor to keep in mind when evaluating trading strategies but also a difficult one to avoid. The causes of this bias are sometimes tricky to spot, and a one-size-fits-all type of recipe for avoiding it is not available. Expertise and knowledge of the matter at hand are essential for developing adequate sampling processes, which only makes it more interesting to work as a researcher!