Survivorship Bias

What is Survivorship Bias?

The failure to account for survivorship bias is probably the most common and significant error made when backtesting portfolio systems on equities. There are currently over 7,000 listed stocks in the U.S. However, there are well over 20,000 stock symbols that have been listed going back in history. The current 7,000 symbols are the “survivors” and any backtest performed on the current stocks excluding the delisted and merged companies will have survivorship bias. Market Testing Wizards only uses high quality data that is “survivorship free” (i.e. includes the delisted company symbols).

Survivorship When Testing Index Components

Survivorship bias problems also show up when a trader attempts to test a strategy on index component stocks. The index components change over time, so if you are testing the current symbols only you are biasing your results to the surviving companies.

How Does Survivorship Bias Affect Backtest Results?

Survivorship bias almost always will have a significant effect on trading system backtests and analysis. Unfortunately it is impossible to predict whether the effect will be positive or negative. In most cases the bias will make bad systems look good, but in some cases good systems can be made to look bad. Either case will result in costly errors. The following table shows three different systems tested first with data from a leading stock analysis software provider and then tested again with “survivorship free” data.

Graphic Table

The breakout system looked very good with standard data, but the inclusion of delisted stocks reduced the returns by almost 40% while increasing drawdowns. Conversely, the momentum system looked very bad with standard data and the concept would likely be discarded; however, with survivorship free data the idea looks like it holds some promise. In the contra-trend system the drawdowns were almost doubled when the delisted and merged company symbols where included. Are you Survivorship Free?