Why Building An Algo Trading System Is More Than Just Testing And Optimization
It happens to every trader. The sudden serendipitous rush of a brilliant trading idea. Maybe it occurs on the drive home after a long day at work. Or, possibly an idea suddenly manifests during a morning shower, and sometimes before falling off to sleep. Wherever it takes place, the normal instinct of the trader is to rush to a computer to test the idea, quickly analyze the results, and if the idea makes money, begin trading it.
Unfortunately, that is the absolute worst approach to take. Even though trading software, with simple strategy development and optimization features, cut the time from creating a trading strategy to seeing the historical results to mere minutes, it does not mean it is the right approach. In fact, a quick test and superficial evaluation is usually the completely wrong way.
To be successful in the long term, the trader must treat strategy creation like building a house. No home builder starts building as soon as he finds a lot and gets some supplies. There are many steps to finish before even starting construction, and the same concept holds true for developing a trading strategy. Creating a plan and building a foundation are two of the first steps a homebuilder, and a trader, must address before any testing and analysis.
Start With A Plan
Ask any builder, and he'll tell you building a house starts with a plan, or blueprint. The blueprint shows what the final product will look like. The same holds true for building a trading strategy; it has to start with a plan, one that shows the end result. Sadly, many people feel that a plan means "find a trading system that makes a lot of money." While that is better than no plan at all, it is too general. How much is "a lot of money?" Being specific, not vague, is the key here.
To be successful in creating trading strategies, a trader first needs to have a detailed plan, goal or vision for the expected performance of the strategy. This way, as the trader develops the strategy, he'll have ways to measure progress. Thus, good strategy development starts with a solid goal. Personal development coaches frequently talk about SMART goals, and that is a good approach to follow. SMART, as shown in Figure 01, is an acronym to help one remember the important features of an outstanding goal: Specific, Measurable, Achievable, Relevant, and Time bound. So, what is an example of a SMART goal for developing a trading system?
Unfortunately, that is the absolute worst approach to take. Even though trading software, with simple strategy development and optimization features, cut the time from creating a trading strategy to seeing the historical results to mere minutes, it does not mean it is the right approach. In fact, a quick test and superficial evaluation is usually the completely wrong way.
To be successful in the long term, the trader must treat strategy creation like building a house. No home builder starts building as soon as he finds a lot and gets some supplies. There are many steps to finish before even starting construction, and the same concept holds true for developing a trading strategy. Creating a plan and building a foundation are two of the first steps a homebuilder, and a trader, must address before any testing and analysis.
Start With A Plan
Ask any builder, and he'll tell you building a house starts with a plan, or blueprint. The blueprint shows what the final product will look like. The same holds true for building a trading strategy; it has to start with a plan, one that shows the end result. Sadly, many people feel that a plan means "find a trading system that makes a lot of money." While that is better than no plan at all, it is too general. How much is "a lot of money?" Being specific, not vague, is the key here.
To be successful in creating trading strategies, a trader first needs to have a detailed plan, goal or vision for the expected performance of the strategy. This way, as the trader develops the strategy, he'll have ways to measure progress. Thus, good strategy development starts with a solid goal. Personal development coaches frequently talk about SMART goals, and that is a good approach to follow. SMART, as shown in Figure 01, is an acronym to help one remember the important features of an outstanding goal: Specific, Measurable, Achievable, Relevant, and Time bound. So, what is an example of a SMART goal for developing a trading system?
The "S" in SMART means the goal has to be Specific. "Developing a trading system that makes a lot of money" is not at all specific. A trader with a non-specific goal will never know when the goal is reached. Instead, a goal like "this trading system needs to average $12,000 net profit per year per contract, over at least 7 years of walkforward testing" is certainly specific enough. Being specific, then, achieves two goals: first, it helps the trader determine when the target is reached, and second, it helps filter out systems that do not meet the plan, before too much time is wasted on an underperforming strategy.
Maybe your goal is 100% annual returns? Can it actually be done? Maybe you should find out before this becomes your goal...
Measurable is the "M" in SMART. Simply put, it means having objective numbers and performance metrics in the plan that the strategy has to meet. It is fairly easy to create a strategy that "minimizes drawdown," but it is far more difficult to create one with "30% maximum drawdown, measured on an trade close to close basis."
"A" stands for Achievable, and this is where a lot of traders go astray. Developing a trading system that "earns at least 20 points per day per contract in the mini S&P futures" is certainly specific and measurable, but it is far from achievable, at least for most traders. Setting an unrealistic goal only leads to frustration, and inevitably causes developers to shortcut the development process. The point is to aim high, but also to aim for something reasonable.
The "R" in SMART stands for Relevant. For trading system design, the trader has to make sure that details in the plan help lead to creating a solid system. If a plan includes "no more than 3 consecutive losing trades," ask "is this criteria truly relevant to developing a trading strategy?" While it may be a nice feature of a trading approach - who would not like a system that never had more than three losses in a row - it really takes away from the primary focus. Remember that a trading plan has to be relevant to creating a long term profitable trading strategy, first and foremost. Other, non-critical items can be put on a "wish list," but should not be the primary focus during strategy development.
The final letter in SMART, the "T," stands for Time bound. Just as no home builder wants to spend 10 years building a house, no trader wants to spends years and years developing a trading strategy. So, good developers put time limits on strategy creation. This is appropriate for two reasons. First, setting a time limit prevents the developer from continuously tweaking and altering a strategy in order to improve it. That type of approach almost always leads to overfitting. Second, time limits keep the developer engaged. Instead of focusing on just one strategy, the developer knows that dozens of potential strategies are waiting to be evaluated, so the focus is then on evaluating ideas, not perfecting one system.
A Solid Foundation
Once the strategy development plan and goal, created using the SMART goal process, is complete, it is time to start the detailed work. For the house builder discussed earlier, the plan is his blueprint, and his construction always starts with the foundation. For a trading strategy developer, the foundation can be considered the strategy building process.
Ask any successful home builder, and he'll tell you that a sturdy house starts with a solid foundation. A house built on quicksand might look very appealing at the beginning, but over time the house will shift and fall apart. The same holds true with trading strategies; without a solid development process, any strategies created will eventually fall apart.
An example of this is shown in Figure 02. Two trading strategies are shown, and both look acceptable, with strategy #1 being vastly superior. Of course, looks can be deceiving, and that is certainly the case here. Strategy #1 was built by optimizing all available data, and by overfitting the rules to the data. In other words, it is underpinned by a weak development process, a weak foundation. Strategy #2, on the other hand, was built with a strong foundation, consisting of limited rules, walkforward testing and sparse optimization. It is therefore much more likely to stand the test of time. So, just seeing an equity curve is not enough - without knowing how it was developed, a trader will never know how realistic the equity curve really is. Knowing what the foundation is made of is critically important.
So, how does a trader create a strong foundation for developing trading systems? First, he has to eliminate all sources of bias, by testing every strategy using the exact same process. "Pet" ideas have to go through the same development process as all the other ideas. This way, all strategies are subjected to the same tests and analysis. The best ideas will naturally rise to the top, without bias. Second, the process has to employ objective performance criteria, and be consistently used by the developer.
One proven trading development process is shown in Figure 03.
This process consists of the following steps:
Trading Idea - A good trader is continuously on the lookout for new ideas to test, and new concepts to analyze. Data Mining and brainstorming are two good sources of new ideas.
Limited Feasibility Testing - Most new traders apply a proposed strategy to all the historical data, typically with excessive optimization. A better way is to test the strategy on a small piece of data. If strategy performance with a small sample is poor, chances are it won't be good on a larger piece.
In-Depth Walkforward Testing, Optimization - Walkforward testing utilizes both optimization and out of sample testing, combined in a way that leads to better real time performance. This approach is vastly superior to traditional optimization, or testing with a single small out of sample period.
Monte Carlo Simulation - History never repeats itself, so it is important to run random number simulations of expected strategy performance. By doing so, a trader can understand the probabilities of achieving a certain rate of return, or of having to endure a certain drawdown. Over on my Calculator page, you can download a Monte Carlo spreadsheet I use everyday - for free!
Incubation/ Initial Testing - After development is complete, it is best to let a strategy sit for a while, before dedicating capital for trading. Monitoring a strategy in real time, but without actually trading, can save a trader thousands of dollars over the long haul, since development mistakes frequently reveal themselves in this step.
Full Implementation - Once everything is in place, full size, real money trading can commence. This stage will also include rules for increasing position size should trading go well, and reducing position size when the strategy has poor performance.
The process above can vary from trader to trader. The important point is that the process exists and is written down. This prevents a trader from bypassing the steps for certain favored strategies.
Conclusion
Developing strategy performance goals, and a strategy development process, are easy concepts to understand. The trick, however, is for the trader to be methodical and rigid in his approach while employing these concepts. Traders who take shortcuts, or who cheat their way through the process, will almost never succeed. Like a house built on sand, a trading strategy developed incorrectly will quickly crumble and fail as the real world unmercifully attacks it. On the other hand, a solid trading strategy, built the proper way, stands a much greater chance of surviving the ravages of the markets.
Where to go next? My article "Building an Algo Trading System - After Testing and Optimization" is an excellent next step.
Trading Idea - A good trader is continuously on the lookout for new ideas to test, and new concepts to analyze. Data Mining and brainstorming are two good sources of new ideas.
Limited Feasibility Testing - Most new traders apply a proposed strategy to all the historical data, typically with excessive optimization. A better way is to test the strategy on a small piece of data. If strategy performance with a small sample is poor, chances are it won't be good on a larger piece.
In-Depth Walkforward Testing, Optimization - Walkforward testing utilizes both optimization and out of sample testing, combined in a way that leads to better real time performance. This approach is vastly superior to traditional optimization, or testing with a single small out of sample period.
Monte Carlo Simulation - History never repeats itself, so it is important to run random number simulations of expected strategy performance. By doing so, a trader can understand the probabilities of achieving a certain rate of return, or of having to endure a certain drawdown. Over on my Calculator page, you can download a Monte Carlo spreadsheet I use everyday - for free!
Incubation/ Initial Testing - After development is complete, it is best to let a strategy sit for a while, before dedicating capital for trading. Monitoring a strategy in real time, but without actually trading, can save a trader thousands of dollars over the long haul, since development mistakes frequently reveal themselves in this step.
Full Implementation - Once everything is in place, full size, real money trading can commence. This stage will also include rules for increasing position size should trading go well, and reducing position size when the strategy has poor performance.
The process above can vary from trader to trader. The important point is that the process exists and is written down. This prevents a trader from bypassing the steps for certain favored strategies.
Conclusion
Developing strategy performance goals, and a strategy development process, are easy concepts to understand. The trick, however, is for the trader to be methodical and rigid in his approach while employing these concepts. Traders who take shortcuts, or who cheat their way through the process, will almost never succeed. Like a house built on sand, a trading strategy developed incorrectly will quickly crumble and fail as the real world unmercifully attacks it. On the other hand, a solid trading strategy, built the proper way, stands a much greater chance of surviving the ravages of the markets.
Where to go next? My article "Building an Algo Trading System - After Testing and Optimization" is an excellent next step.
About The Author: Kevin Davey is an award winning private futures, forex and commodities trader. He has been trading for over 25 years.Three consecutive years, Kevin achieved over 100% annual returns in a real time, real money, year long trading contest, finishing in first or second place each of those years.
Kevin is the author of the highly acclaimed book "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: https://kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.
Kevin is the author of the highly acclaimed book "Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading" (Wiley 2014). Kevin provides a wealth of trading information at his website: https://kjtradingsystems.com
Copyright, Kevin Davey and KJ Trading Systems. All Rights Reserved. Reprint of above article is permitted, as long as the About The Author information is included.