I saw an advertisement for a futures trading system the other day, and I’d thought I would discuss it here. Now normally, I trade only futures, forex and commodity systems I create, since I know exactly what goes in them, how they were tested, etc. That’s not to say there aren’t other vendors and systems out there that are good. Some are very good, and worth checking into.
I usually scan ads to see what other system vendors in the industry are up to, and alert my readers and subscribers of some of the tricks to be on the lookout for. I’m going to share one of those sneaky tricks in a minute.
This ad caught my eye because it showed tremendous past performance, such as the ability to make hundreds to thousands of dollars per day on 1 or 2 mini S&P contracts. Performance so good that it makes you wonder why the vendor is selling it, but that’s another story…
Enticed by the ad, I clicked on through to show hypothetical performance data. In the vendor’s mind, once I saw how good the system was at picking trades – how well it picked tops and bottoms – I would undoubtedly be hooked. Unfortunately for them, it had the opposite effect.
Why would hypothetical price charts that showed how well a system picked tops and bottoms work against the vendor? Simple: incorrect backtesting. Please let me explain.
The system in question showed many trades where the system bought at the absolute low of a bar, and sold at the absolute high of another bar. Since there is a bid/ask spread in every futures market, it is normally impossible, or darn near impossible, to get filled on a buy order at the exact lowest price (and the same for selling at the high). You might get filled once in a while at that price, but not consistently.
But when a vendor backtests incorrectly, the system can hypothetically get filled EVERY time at the low (for buys) or at the highs (sells). Those are the results you’ll see in the system performance report. In real life, you’d only get filled on the losing trades, but only rarely on the winning trades. So, your results won’t be nearly as good as the hypothetical, vendor reported results. In fact, many times correcting for this trick turns a winning system into a real life losing system – it is potentially that bad.
Confused? Well, it is a complicated topic, one most people would not even recognize. But here is what to look for: If hypothetical results show the system can consistently buy the absolute low and sell the absolute high (the extreme points) of a price bar, chances are the system will not work in real time.
The lesson here: make sure you look at any performance history with a skeptical eye. Doing so might save you a ton of money.