In recent months I’ve been working with planners over a building plot. Anyone who’s ever had to deal with local planning authorities will know what I’m talking about …
You look back over what’s been approved in the past, you collect all the information you possibly can, you talk to everyone “in the know” – and formulate a plan …
But, inevitably, the goal posts will have moved since your last conversation two weeks ago … the agenda has changed … and all your well-thought-out plans end up in the bin.
It would seem that our local planners have taken a lesson or two in unpredictability from the markets …
“Past performance is no guarantee …”
It’s a phrase we probably all recognize: “Past performance is not necessarily a guide to future performance.”
It’s a requirement of the FSA that this is pointed out whenever a firm shows historical information on an investment opportunity.
So, just as I feel put out when I building scheme I come up with – that was approved down the road five years ago – gets thrown out, the market’s, too, can’t be relied on for consistency.
Looking at the way markets have behaved in the past, alone, is no way to come up with a successful trading strategy. Yet many such strategies make claims purely on back testing alone – when they should also be forward testing.
Re-writing history: the pros and cons of back testing and forward testing
Back testing refers to applying a trading system to historical data to verify how it would have performed in that period.
Because we know what happened in the past, it’s easy to come up with a trading system that would have been very profitable in that time frame.
With the software I have on my computer, it probably wouldn’t take me too long to come up with a trading strategy that would look like a great money-maker, based on purely back-tested results.
We could run through all the charts, and find the optimum trading times, hone our targets and stops around actual levels that the charts hit, add in some rules to explain away instances when the markets misbehaved for us …
Hey presto! The next get-rich-quick scheme.
Leap of faith
Any trading system relying on backtested results alone suffers from considerable limitations:
* Many system developers misuse the “optimization” feature on testing software. This allows the software to select the best set of parameters based on past data – there is no reason to believe that these figures will be the same in the future.
* Even the best-intentioned of us find it hard not to apply hindsight to a backtest. We know past market data, and it is very hard to take this out of the equation – even on a sub-conscious level – when backtesting.
* Often backtested data fails to take into account issues like slippage, dividend adjustments, the spread, and rolling charges.
For all these reasons, it’s very important to take any backtested results with a large tablespoon of salt. An unprofitable system can be magically tweaked into a profitable one with a few simple adjustments.
So, what can we trust?
In-sample and out-of-sample – beating the optimizers
One way to beat the software optimization is to take two backtest periods. On the first period, you’ll apply optimization – this will give you the parameters for your strategy.
In the second backtest period, you don’t apply optimization.
An indication that the strategy will have success in the future would be shown by a strong correlation between the results of the two periods.
Walking forward
If backtesting is the wayward prodigal son, then his sensible sibling would be “forward testing”.
There are a couple of issues with forward testing, that I’ll explain in a moment, but first, here are the key factors that set it apart.
Forward testing takes the parameters of the trading strategy and applies them to a live market. It is sometimes called paper trading, as the trades are recorded on paper, rather than with real money.
Forward testers still need to be mindful of applying hindsight – it’s too easy to fall into bad behaviours of cherry-picking trades and rationalizing that “I’d never have placed that trade, because ….”
Many traders use a demo account to forward test strategies. Demo accounts are a great way of getting used to a strategy and testing its viability, as they will include the spread, slippage, dividends, and financing costs. However, you should be aware that prices do not always match those on live accounts, so results can differ.
Worth the paper it’s written on
Now, I mentioned above that there were still pitfalls with forward testing …
* Forward testing can be subject to some cherry picking of successful trades, so needs to be undertaken with firm discipline.
* Forward- and back-tested results often don’t include the additional costs that can make an enormous difference to the profitability of a strategy.
* But probably the biggest problem with forward testing is that – like many sensible behaviours in life – it’s very, very boring. Which is why so many traders skip this step, or fudge it in just a handful of weeks.
For all these reasons, it’s important to look for forward-tested results and to assess the effect of financing costs on those results (assuming these costs aren’t already countered in).
If these results are independently verified – so much the better.
Independent witness
There are a handful of companies that offer independent verification of the forward-tested results of trading strategies.
Probably the best-known of these is collective2.com
Collective2.com acts as an online repository for automated trading systems. You can view results from thousands of trading systems online, allowing you to pick and choose the ones you like.
It’s worth noting here, that these results still don’t include the spreads and financing costs – so these will have to be factored in.
These services also need to make their money. Normally the system developer will give up a cut of their subscription fees, but the money to cover this cost has to come from somewhere.
Of course, my personal favourite for a truly independent and knowledgeable view on trading strategies is to run it by our man Dave Evans in What Really Profits.
In the next seven days …
Next week’s hot ticket will be the budget on Tuesday. Investors will be watching the gilt market closely, and Mr Osborne will hope to see yields fall as the UK debt looks less risky.
At 7.15pm UK time on Wednesday, we have the FOMC statement on interest rates. The wording of the statement will be key, with more hawkish members calling for the removal of the words “extended period of time of very low rates” from the statement.
