‘Past performance is no guarantee…’
It’s a phrase we probably all recognize: ‘Past performance is not necessarily a guide to future performance’.
It’s a requirement of the FSA that this is pointed out whenever a firm shows historical information on an investment opportunity.
So, just as I feel put out when a building scheme I came up with that was approved down the road five years ago now gets thrown out, the markets too can’t be relied on for consistency.
Looking at the way markets have behaved in the past alone, is no way to come up with a successful trading strategy.
‘Back-testing’ refers to applying a trading system to historical data to verify how it would have performed in that period. ‘Forward-testing’ is applied to a live market.
Back-testing and forward-testing trade strategies explained
Let’s take a look at each in turn…
Re-writing history
Because we know what happened in the past, it’s easy to come up with a trading system that would have been very profitable in that time frame.
With the software I have on my computer, it probably wouldn’t take me too long to come up with a trading strategy that would look like a great money-maker, based on purely back-tested results.
We could run through all the charts, find the optimum trading times, hone our targets and stops around actual levels that the charts hit, add in some rules to explain away instances when the markets misbehaved for us…
Hey presto! The next get-rich-quick scheme.
Leap of faith
Any trading system relying on back-tested results alone suffers from considerable limitations:
* Many system developers misuse the ‘optimization’ feature on testing software. This allows the software to select the best set of parameters based on past data – there is no reason to believe that these figures will be the same in the future.
* Even the best intentioned of us find it hard not to apply hindsight to a back-test. We know past market data and it is very hard to take this out of the equation – even on a sub-conscious level – when back-testing.
* Often back-tested data fails to take into account issues like slippage, dividend adjustments, the spread, and rolling charges.
For all these reasons, it’s very important to take any back-tested results with a large tablespoon of salt. An unprofitable system can be magically tweaked into a profitable one with a few simple adjustments.
So, what can we trust?
In-sample and out-of-sample – beating the optimizers
One way to beat the software optimization is to take two back-test periods. On the first period, you’ll apply optimization – this will give you the parameters for your strategy.
In the second back-test period, you don’t apply optimization.
An indication that the strategy will have success in the future would be shown by a strong correlation between the results of the two periods.
Walking forward
If back-testing is the wayward prodigal son, then his sensible sibling would be ‘forward-testing’.
There are a couple of issues with forward-testing that I’ll explain in a moment, but first, here are the key factors that set it apart.
Forward-testing takes the parameters of the trading strategy and applies them to a live market. It is sometimes called paper trading, as the trades are recorded on paper, rather than with real money.
Forward-testers still need to be mindful of applying hindsight – it’s too easy to fall into bad behaviours of cherry-picking trades and rationalizing that, “I’d never have placed that trade, because…”
Many traders use a demo account to forward-test strategies. Demo accounts are a great way of getting used to a strategy and testing its viability, as they will include the spread, slippage, dividends, and financing costs. However, you should be aware that prices do not always match those on live accounts, so results can differ.
Worth the paper it’s written on
Now, I mentioned above that there were still pitfalls with forward-testing.
* Forward-testing can be subject to some cherry picking of successful trades, so needs to be undertaken with firm discipline.
* Forward and back-tested results often don’t include the additional costs that can make an enormous difference to the profitability of a strategy.
* But probably the biggest problem with forward-testing is that – like many sensible behaviours in life – it’s very, very boring. Which is why so many traders skip this step, or fudge it in just a handful of weeks.
For all these reasons, it’s important to look for forward-tested results and to assess the effect of financing costs on those results (assuming these costs aren’t already countered in).
If these results are independently verified, so much the better.
Independent witness
There are a handful of companies that offer independent verification of the forward-tested results of trading strategies.
Probably the best-known of these is collective2.com
Collective2.com acts as an online repository for automated trading systems. You can view results from thousands of trading systems online, allowing you to pick and choose the ones you like.
It’s worth noting here that these results still don’t include the spreads and financing costs – so these will have to be factored in.
These services also need to make their money. Normally the system developer will give up a cut of their subscription fees, but the money to cover this cost has to come from somewhere.
Of course, my personal favourite for a truly independent and knowledgeable view on trading strategies is to run it by our man Dave Evans in ‘What Really Profits’.
