Survivorship bias
Data sets of today’s stocks forget the ones that failed. · 7 min
ProOpen to everyone while Pro is in preview.
Download the stocks in an index today and test a strategy on their last ten years: the companies that went bankrupt, were delisted or shrank out of the index are missing. The test only sees survivors, and survivors are winners by selection.
Predict first
Forty simulated stocks over five years. A strategy buys any stock that has fallen 50% from where it started and holds to the end. Tested only on stocks still listed at the end, will it look better or worse than on all forty?
The stocks that vanished
Forty simulated stocks over five years. A stock that falls below a quarter of its first price is delisted, and a data file of today’s stocks no longer contains it.
Running the numbers…
In the default universe, 16 of 40 stocks were delisted. Buy-and-hold on the survivors returned −3.9%; on every stock listed at the start, −32.9%. The difference is not skill or strategy: it is who was left out.
Worked example
Three stocks start at $10. After five years: $14, $9 and delisted at $1.50.
- Survivors only: (+40% − 10%) ÷ 2 = +15% average.
- All three: (+40% − 10% − 85%) ÷ 3 = −18.3% average.
Common mistake
Testing on today’s index members. Use point-in-time membership: the stocks that were in the index on each historical day, delisted ones included, with their final prices. Strategies that buy losers (dip buying, value, mean reversion) are the most inflated by survivor data.
Closing round
Find the flawed backtest
Three reports on the same ten years. One of them could never be traded as written.
Building a fresh case…
Further reading: Aronson, Evidence-Based Technical Analysis, chapter 1; Davey, Building Winning Algorithmic Trading Systems.