A Comprehensive Guide to Backtesting Vs Forward Testing
Walkforward Testing - Pros and Cons for Algo Trading
Updated January 2023
Backtesting, forward testing and walkforward testing are three methods used by traders to evaluate the performance of trading strategies. Before deciding which method to use, it is important to understand how each works and what benefits and risks they offer.
If you've been around algo trading for a while, you probably have seen trading system vendors tout their performance results via "backtest" or "walkforward test" or "forward test" reports. Unfortunately, most people think these three types of tests are the same, or virtually the same. Nothing could be further from the truth.
My choice of testing for algo strategies is walkforward testing, but before I get into some of the details of walkforward, let's define and examine all three types of testing.
Backtesting, forward testing and walkforward testing are three methods used by traders to evaluate the performance of trading strategies. Before deciding which method to use, it is important to understand how each works and what benefits and risks they offer.
If you've been around algo trading for a while, you probably have seen trading system vendors tout their performance results via "backtest" or "walkforward test" or "forward test" reports. Unfortunately, most people think these three types of tests are the same, or virtually the same. Nothing could be further from the truth.
My choice of testing for algo strategies is walkforward testing, but before I get into some of the details of walkforward, let's define and examine all three types of testing.
What Is Backtesting?
Backtesting is a method of using historical data to validate the effectiveness of a trading strategy. This is done by running trading rules (a trading algo), with the historical data being used as input for the model or algorithm.
The results of this simulation are then used to measure the performance of the strategy. As backtesting does not require any real-world trades, it can be done quickly and cheaply, making it ideal for testing new ideas.
The results of this simulation are then used to measure the performance of the strategy. As backtesting does not require any real-world trades, it can be done quickly and cheaply, making it ideal for testing new ideas.
What Is Forward Testing?
Forward testing is the process of using real-world and real time trading data to test hypothetical trading strategies. This type of testing involves actually placing the trades that are generated by the trading strategy and observing the results in real time. Usually this is conducted in a real money account, BUT it can be done in a simulator, IF the simulator is of high enough quality and replicates real world market conditions. Most simulators are not that accurate, though.
This test offers a more accurate representation of how successful a strategy will be across different market conditions, as it includes aspects such as impact costs, liquidity, and volatility. There really is no better test for seeing if a strategy works in real time.
This test offers a more accurate representation of how successful a strategy will be across different market conditions, as it includes aspects such as impact costs, liquidity, and volatility. There really is no better test for seeing if a strategy works in real time.
What Is Walkforward Testing?
Walkforward testing is a type of backtest. I discuss it in detail below, but when it is done correctly, it eliminates many of the issues and pitfalls inherent in normal backtesting. It produces accurate "out-of-sample" results based on historical trades.
Advantages of Backtesting
Backtesting offers the advantage of being able to test many combinations of parameters and historic market conditions in a short amount of time. With enough backtesting data, traders can develop robust trading systems and control for risk-reward ratios.
However, backtesting is limited by the fact that historical data may not accurately reflect current market conditions, as markets are constantly evolving. Therefore, it is important to complement backtesting with forward testing to get a complete picture of how a particular strategy will perform in real-world trading.
However, backtesting is limited by the fact that historical data may not accurately reflect current market conditions, as markets are constantly evolving. Therefore, it is important to complement backtesting with forward testing to get a complete picture of how a particular strategy will perform in real-world trading.
Advantages of Forward Testing
One of the primary advantages of forward testing is the ability to see how a strategy performs in real-time market conditions. This can provide traders with a better understanding of how their trading strategies will perform when they’re actually put into action.
Additionally, forward testing also allows traders to tweak and refine strategies as necessary, in order to optimize performance (although this can be REALLY dangerous). On the other hand, backtesting is limited by its reliance on historical data, which can make it difficult to gain an accurate picture of real-world performance.
Additionally, forward testing also allows traders to tweak and refine strategies as necessary, in order to optimize performance (although this can be REALLY dangerous). On the other hand, backtesting is limited by its reliance on historical data, which can make it difficult to gain an accurate picture of real-world performance.
Advantages of Walkforward Testing
The biggest advantage with walkforward testing is that it avoids a lot of the over optimization problems in traditional backtesting. This is discussed in more detail below.
Best Practices for Backtesting, Forward Testing and Walkforward Testing
Ultimately, it’s important to incorporate both backtesting (via walkforward testing) and forward testing into your trading strategy development process.
When conducting walkforward backtesting, be sure to use large amounts of data, so that you have a good grasp of the historical trends in the markets. But, be careful not to overtest - that is something that walkforward testing avoids (when it is done correctly).
Additionally, when conducting forward testing, be sure to use all available tools like charting software and analysis programs to help take emotions out of the trading decision-making process. A good testing process will not eliminate emotion (that is a common misconception), but it will help reduce and mitigate emotions from your algo trading.
Finally, be sure to establish risk tolerance limits before beginning any tests. This is key no matter how you test!
When conducting walkforward backtesting, be sure to use large amounts of data, so that you have a good grasp of the historical trends in the markets. But, be careful not to overtest - that is something that walkforward testing avoids (when it is done correctly).
Additionally, when conducting forward testing, be sure to use all available tools like charting software and analysis programs to help take emotions out of the trading decision-making process. A good testing process will not eliminate emotion (that is a common misconception), but it will help reduce and mitigate emotions from your algo trading.
Finally, be sure to establish risk tolerance limits before beginning any tests. This is key no matter how you test!
My Choice - Walkforward Testing Is Best
So, what is the big difference between backtests and walkforward ("out of sample") tests? In a nutshell, with traditional backtest optimizations you take all data, optimize your parameters and then take the best parameters and start trading it. Traditional backtesting produces an excellent looking, almost-to-good-to-be-true equity curves for the past, but rarely works as well going forward. This method is what you typically see in ads, proffered many times by unscrupulous algorithmic trading system sellers.
Walkforward, on the other hand, takes a small chunk of data, optimizes the parameters, and then applies those values to the next chunk of UNTESTED data. Those new results become part of your final results, one piece at a time. Since walkforward testing calculates the performance on untested and unoptimized data, walkforward is a truer indication of future performance.
If you are looking at any track record, first and foremost, look for real time, audited results or those linked to actual trading accounts. Nothing is more realistic than that. If actual results are not available, walkforward results are the next best thing. Compared to real time performance, walkforward results will usually be somewhat optimistic, however.
Backtest results, on the other hand, are typically so full of hindsight, over-optimization and curve fitting that these results should be VERY carefully scrutinized. Many times, backtest results are so full of intentional or unintentional errors that the results are worse than useless. If the backtest results look too good to be true, they probably are.
So, be careful when looking at performance results of ANY algorithmic trading system or strategy.
Take the time to understand what you are looking at, and how it was created. Look for real time results first, then walkforward results. Beware of backtest results.
Here's a picture that describes the process:
Here is a nice overview of walkforward testing:
Here is a neat little video I did on 2 different types of walkforward analysis:
And here is a video on manual walkforward analysis:
HTML Comment Box is loading comments...
About The Author: Kevin Davey is an award winning private futures, forex and commodities trader. He has been trading for over 25 years.Three consecutive years, Kevin achieved over 100% annual returns in a real time, real money, year long trading contest, finishing in first or second place each of those years.
Kevin is the author of the highly acclaimed best trading book "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: http://www.kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.
Kevin is the author of the highly acclaimed best trading book "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: http://www.kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.