Avoid These 7 Options Backtesting Pitfalls

Abstract tech pattern

Backtesting is a simulation of an investment strategy in the past that seeks consistent trading performance in the future. It can help develop and scrutinize the performance of systematic options strategies. The complexity of options market can be simplified by defining rule-based strategies and observing the backtest outcomes.

This makes it an important tool, especially for options traders. However, there are some shortcomings investors need to be aware of. Here's how to understand these pitfalls, as well as solutions for avoiding them.

Pitfall 1: Over-reliance on backtesting

Starting to trade based only on a backtest is premature. More needs to be done. Many backtested strategies do not live up to expectations. They perform well in the backtest, but do not perform as well when implemented.


Testing out of sample and paper trading against live markets, and seeing consistency with backtest, provides more confidence in the strategy.

Pitfall 2: Overfitting the historical market data

As more strategies are tested on the same dataset, there is a greater risk of uncovering a lucky strategy, which lacks consistent performance out of sample or in actual trading.


Attempt to cross validate out of sample and with other similar tickers. Cross validation of the factors in the test by isolating these parameters and analyzing performance.

Pitfall 3: Data mining, correlation and causation

Using massive amounts of data and identifying spurious relationships will not be consistent when going forward and implementing a strategy. Finding causation is good but finding correlation instead of a causation will also create inconsistent results going forward.


Understand the source of alpha, understand the data and consider more stringent tests and thresholds for significance. An options strategy is trading based on predefined rules. The strategy can be based on fundamental, technical or options information, but needs to be quantifiable information that makes sense to the task at hand. Dependent variables need to have some reasonable relationship to dependent variables and have enough differentiated data for backtesting with accuracy and rigor.

The job of backtesting is to find opportunities that exist. These opportunities need to be understandable and justifiable. Here are some examples of opportunities that may exist:

  • Psychological trading anomalies: Anomalies in the market may present themselves as a result of such fallacies of sunk costs or the disposition effect of holding stocks that have lost money and selling those that have been profitable.
  • Asymmetry of information: Opportunities may exist from finding where people in the know tip their hand when they trade on asymmetric information. 
  • Risk tolerance differences: Options may be over or underpriced for exogenous reasons like bank risk mitigation and may present an opportunity to those with differing risk preferences.
  • Tax idiosyncrasies: Dividend preferences based on tax rules may present mispricing for those with different tax treatment.
  • Barriers to entry: Some investments may be unavailable to traders. Information on the inner workings of ETFs might be understood better by various traders. Understanding and gaining access to such markets may be advantageous.

Pitfall 4: Hidden risks

Some options positions have nonlinear risk as the underlying moves against the position.


Know your risks by simulating a position and stress testing. Use appropriate slippage and commission assumptions. A major risk in backtesting is assuming where to trade. Especially in wide bid-ask spreads, trading based on mid-market will not be where the trade can be filled. Avoid wide markets in backtesting.

Pitfall 5: Over-focus on return

By narrowing in on annual returns, many other important statistics can be missed.


Use other statistics and temper return results:

  • Sharpe ratios-- annual return divided by volatility
  • Sortino ratios-- annual return divided by volatility when returns go against the position
  • Drawdowns-- amount lost from peak to valley
  • Win rates-- percent of winning trades to losing trades
  • Best/worst month/year-- seeing outlying returns in various timeframes

These and more are important statistics to understand the strategy's behavior.

Pitfall 6: Selecting an inappropriate time period

Testing on a time when the market was in an environment that is different than are likely to repeat, will cause inconsistent and misleading expectations.


Make sure there is enough history to contain a representative set of environments. For example, market data should go back to 2007 to include the 2008 market downturn. Out of sample testing should also have a variety of environments.

Pitfall 7: Survival bias

Some options strategies rely on tickers that may not have been in existence or may have been delisted. Including delisted tickers can affect the results greatly.


Include delisted tickers in the backtest. Only trade indexes or ETFs that are likely to exist for a long time.

Final note

Avoiding pitfalls in backtesting can produce a higher probability of consistent outcomes when implementing trading strategies. A rules-based strategy can help the trader avoid behavioral biases and help take advantage of structural anomalies in the market. It is important to estimate and understand why the alpha exists and if the source is sustainable. Backtesting a rules based strategy that avoid pitfalls is a great tool for traders.


Matt Amberson

Matt Amberson, Principal and Founder of Option Research & Technology Services. ORATS was born out of a need by traders to get access to more accurate and realistic option research. Matt started ORATS to support his options market making firm where he would hire statistically minded individuals, put them on the floor, and develop research to aid in trading options. He is heavily involved with product design and quantitative research. ORATS offers data and backtesting on a subscription basis at www.orats.com. Matt has a Master’s degree from Kellogg School of Business.

Read Matt's Bio