Those of us who employ mechanical rule based trading systems, as well as those who “trade around” their rule-based systems, are both looking for one very important quality —a strategy that performs as well, or nearly as well, in live trading as the historical back test results indicate.
There is no sugar coating it, this is a difficult quality to realize. However it is also definitely an attainable goal. The following observations provide some solid guide posts for constructing and evaluating a mechanical rule based trading strategy before committing money to it.
The designing of a rule-based trading strategy requires some degree of “optimization” of parameters within the trading rules. These parameters may include technical indicator input values such as relative strength index (RSI) or ADX, time values such as changing the exit rules after “n” bars, or price functions such as stop loss and profit objectives. Optimization of a trading strategy is necessary to determine precisely what is valid and what is not. Every trading strategy must therefore be optimized to some extent.
Of course, the most significant risk in any trading strategy development is “over-optimization,” also referred to as “curve fitting.” This process essentially mines historical data and attempts to identify rules and parameters that fit past data. Over-optimized trading strategies are typically characterized by at least one of (i) too many rules or conditions (ii) too few trade occurrences per rule (iii) erratic results with small adjustments in input parameters. Over-optimization will result in trading strategies where the historical performance is vastly superior to subsequent live trading.
To avoid the inherent risk of over-optimization in developing Trading Strategies, consider implementing these “8 Simple Rules” of Strategy Development:
#1 Cut The Number Of “And” Statements
Use only one or two “and” statements per entry or exit rule. For example, if precondition 1 exists and Momentum > signal threshold then buy.
There are reasonable exceptions to using more then a single “and” statement rule. For example, if precondition 1 exists and momentum > signal threshold and “close > moving average” then buy. In this case we have added a long-term trend filter in the form of “close > moving average” with the additional “and” statement. A “trend filter” simply means only trade long if the trend is up as identified by price being greater then or above the moving average. Vice versa for shorts.
Over-optimization occurs when additional “and” statements are added. In our example if we added and “close > than open 2 bars ago”, and “RSI (14) 30 we are definitely into dangerous territory. These conditions will result in fewer and fewer trade occurrences which increasingly moves away from a general discovery of market character, and into the pitfalls of over optimization.
#2 Use A Single Entry Principle
We might have several entry rules but they all use the same entry principle. For example rule #1 might be if precondition 1 exists and momentum > signal threshold then buy. Rule #2 might be if Bollinger Band® Difference signal threshold then buy. Both rules use precisely the same “momentum > signal threshold”, but each supplies different circumstances by which it may be followed.
This approach helps ensure that the strategy generates a significant number of total trades and validates that the entry works under various market conditions.
The more trade occurrences a trading system has, and the more general its application, the greater the probability that one is going down the path of uncovering repetitive market behavior. And markets do repeat certain behaviors, just never in plain sight.
Hence more trades means less chance the test results are based on coincidence due to small sample size. More importantly, the more trade iterations, the harder it is to curve fit or over optimize a signal.
#3 Apply Robust Input Parameters
Any and every input parameter used in a trading strategy is an optimizable parameter. We’re looking for a wide range of input parameters that are satisfactory. We also look for a gradual falling away on each side of the optimum value. Strategies that display this characteristic are referred to as “robust.”
For example in a moving average trend filter we might optimize a moving average length from 10 to 100 by 5’s. If the optimization report resulted in a hierarchy of results where moving average values were 40, 60, 65, 70, 55, 50, 75, 45 etc we would choose “65” from this list as the trend filter. This is because the highest ranking value of 40 appears to be an outlier — it likely by coincidence hit a couple big trades. While 65 is the 3rd best parameter the two adjacent values are the 2nd and 4th best parameter. Thus 65 represents the most robust middle ground for this test.
#4 Rules Are Portable Between Markets
Portability means that the same principles can be generally successfully applied to at least some non related markets. The portability of entry/exit rules between markets reinforces that the rules at work are not curve fit to a particular data set. Absolute identical portability is not necessary, as it is also a characteristic of markets that they have their own personalities.
A principle that works in a general sense across a few different markets is a lot more likely to hold up in the future. This embraces the philosophy that successful rule based trading system design is built around discovering or uncovering how markets work, as opposed to fitting very narrow rules to a single historical data set.
#5 Exit Rules Follow The Same Design Requirements As Entry Rules
All evaluations applicable to entries are equally applicable to exits. Initial tests for any rule based entry should begin with very simple exits such as dollar stop loss, and dollar profit objectives, or volatility stops and objectives. If any particular approach does not show promise with simple exits, then move on. Torturing the data with a lot of different exits to get it to “work” is likely going to lead to over-optimizing and hence disappointment in live trading.
Related to this is all exits used in a trading strategy should be the same for all the related entry signals. If entry rules all follow the same general principle, then the same exits should work with all of them. Mixing and matching different exits with similar entries is very often just another form of over-optimizing.
#6 Review Long and Short Trade Results Separately
Always look at a “Long Only” trades and “Short Only” trades in the evaluation of a trading system. This is how one can see how the system performs within different types of market trends.
For example how does the system do on the short side during a general bullish year? During strong trends in either direction, we want to see our trading strategy minimize loss when it is taking trades against the trend or perhaps even more significantly, against a change in prevailing trend.
The standard is to look for a trading strategy that does not get buried during the inevitable times it falls out of sync with the market, regardless of the trend or lack of a trend. Looking at “Long Only” and “Short Only” separately helps to point out possible issues that can get obscured when looking at only the combined trading system results.
#7 Do Not Optimize Entries And Exits Separately
While it is sometimes true that markets sometimes behave differently on the way up than they do on the way down, and one could definitely optimize to get better historical results on this basis, this approach is step closer to over-optimization. Since the goal is near zero trading strategy degradation post development, always use mirror image signals for either side of the market. As alluded to above, be cautious of “long only” or “short only” trading systems, where the appearance of success may be largely dependent on a very long term secular trend, more than the system itself.
#8 Review the Length of Track Record Several Ways
One very overlooked metric — perhaps the single most overlooked metric in system evaluation — is how often a trading system has a position in the market long or short.
In this regard a system that has a trading position 50% of the time is more difficult to over optimize then one that has a position only 5% of the time. A caution here — a system that is in the market only 5% of the time may still be very valid, albeit just easier to over optimize.
A trading system that has been either long or short 50% of the time through several up and down market cycles is less likely to encounter something in the future that is significantly different then the past. Hence the system is less likely to be over optimized and less likely to substantially degrade going forward.
The other reason for a long test period is that markets do go through cycles where at least some characteristics differ. In the e-mini S&P index for example, the history has 3 distinct market environments; from 1997 inception to roughly 2004, then from 2004 through 2006, and then 2007 to present (2014). If one is running a system that only appears satisfactory from 2007, then that is a big potential red flag. No one knows when another type of market environment might appear. Test over all the history available.
Summary
Over-Optimization is the downfall of rule-based trading strategies and it is not easy to avoid. By following the above guide posts though, one can eliminate much potential disappointment. No doubt these guide posts make it more difficult to develop a decent looking historical test and that is precisely what they are intended to do. If one can follow these guide posts then any trading system will have a much better chance of survival when taking the strategy to live trading.
= = =
Learn more:
www.JDTradingSystems.com
E-Mail: KeyPoint@sympatico.ca
