One of the most difficult things with trading systems is knowing when to stick with a trading system versus making changes or calling it quits.
Of everything I’ve ever read on when to throw in the towel on a system, I like Tom Basso’s thoughts on the subject the most. The following Q & A comes from a recent interview Tom did. You can find a link to the whole interview below.
Q: How do you know when your strategy may be broken versus just being out of favour with the current market?
A: Broken strategies are found when they do something that you didn't design it to do…For example if you are buy and hold long stocks and the stock market goes down 50%, and you lose 45%-60% of your equity, that would be expected and the strategy is not broken. Nothing to fix there. If you are timing using a 20/100 day moving average and the stock market goes through a sideways period of 1 year and ends up unchanged for the year and you are down 10-15%, that's also to be expected with the whipsaws. It's not broken, so don't try fixing it…If [the trading system] is doing what you expected during the conditions that have existed, then it is not broken, so don't be fixing it.
To summarize, just because a system loses money does not mean that it is broken. A trend system in a choppy market will almost certainly lose money just like a mean reversion system will lose money during trending markets. This is a function of the wrong market environment for the system, not any inherent flaw in the system’s design. Conversely, if you have a model that intends to benefit from a rising stock market and it loses money despite the stock market rising, you likely have a truly broken system since it isn’t doing what it is supposed to be doing. The key determinant in declaring a system broken comes down to if the system does what you expect it to do, not whether or not it makes money in any given time period.
Considering the above, asking if a system is broken might actually be the wrong question. The better question might be: will this system fit the coming market environment and make money? Famed systems pioneer/trader Richard Dennis offered some comments on the subject in a 2004 interview in Stocks & Commodities entitled The Legend And The Lore:
Nine out of 10 things that worked 20 years ago don’t work now...That’s what makes it tough... What I try to do now may be a little subjective. I like to find something that worked in the last year when markets were bad, but also worked in the previous five when markets were mediocre, and also worked 15 or 20 years ago...I...change the concept to the point that if you knew what I was doing three years ago, you probably wouldn’t have any clue what I’m doing now. I have to admit it’s subjective. It has to do with trying to anticipate what systems the markets are going to accommodate. I try to come up with novel ideas, ideas that others haven’t run to ground.
In trying to solve for the unknowable future, you have several options. You can diversify across systems be it using multiple timeframes (ex. use 3 breakout models vs just 1) or marrying systems that benefit from different market regimes (i.e., trend systems and mean reversion systems). You might use pattern analysis to attempt to determine what systems “should” work going forward. Or you might attempt to align trading systems with some kind of fundamental driver (ex. the Bank of Japan wants a strong currency). Each of these options has pros and cons but, ultimately, no one knows what is coming. The answer to the question of which system will work best next will only be obvious in hindsight.
In considering all of the above, two quotes come to mind:
This is not a perfect game. - Steve Cohen in Stock Market Wizards
You pays your money and you takes your chances. - Richard Dennis in the S&C Interview
Here is a link to Tom’s interview:
Some words of wisdom right there.
I love your expanding on my thinking. Spot on, George!