Tuesday, June 19, 2012

Step 9 - How To Backtest

Contest Update:

Still underwater (about 10% loss from start).  Still trading exactly as planned (except where initial margin becomes issue).  Still VERY disappointed at the last month of performance.  Still feeling more than a bit depressed and embarrassed at having to show this to the world.

I hope that readers get one takeaway from my current performance: anything can happen in trading.  Historically winning strategies can turn negative (for either a little while, or maybe forever).  Don't position size assuming that you will be profitable - that is trading suicide.


The Trading Process - Backtest the Right Way - Step 09

Last time we discussed "expectancy."  If your trading approach doesn't have a positive expectancy, DON'T TRADE!  It is critical.

Of course, to get historical trades to determine your trading system expectancy, you must backtest.

You can either backtest by hand, or use trading software to help you.  Backtesting can be very tricky, though.  Here are a few tips to help you backtest properly:

1) Make sure you include realistic costs for commissions and slippage.  I typically use $25-50 per contract per round turn trade.  My numbers are based on actual trading.  Many strategies look great with $0 commissions and/or slippage.  THESE SYSTEMS ARE FANTASY!

2) If you use backtest software, learn the tricks and pitfalls in it.  In Tradestation, for example, the default setting for limit orders will assume a limit order is filled if touched.  Traders know that in most instruments, you'll never get filled on a "touch" of your limit price.

3) Make sure you aren't "peeking" into the future.  This is really easy to do if you backtest by hand or with Excel.  An example of this would be buying today's close if tomorrow's open is up - you can;t know this in advance.

4) If your results look too good to be true, they probably are.  You either done something wrong, optimized your backtest too much, or curve fit rules to match the data.

5) Your backtest should definitely include either a significant out of sample test period, or should use walkforward backtesting.  If you don't know why this is important, make sure you research it and learn.  I guarantee that following this step alone will save you thousands.  Only the inexperienced optimize over all their data, and then start trading.

Obviously, proper backtesting is a huge topic.  I've just scratched the surface.

Next: Incubation, not just for baby chicks any more!


  1. Nice post Kevin, a back tested "system" is a must if a trader is to have long term confidence in their method.

  2. Hi Kevin, i was searching your emai address but I was unable to find it, I hope you will find the time to read these few lines. I have being following you in yours ups and downs. In some way they reflected mine. There is no room for Depression and embarassement. After a decade working for investment banks in uk, us and italy i have started my prop trading business and i m trying to find my path. It s hard but we love our job we do it with passion. I have just closed a slight negative month and i m negative since the beginning but i m going forward, we are going forward. I hope to be able to get in touch. You can write me at paco@77sigmatrading.com.

  3. Trin - Without confidence, things get really, really tough. Even with confidence, its not that easy.

    Skater - Thanks for the comment. Good luck with your prop trading. As you know, it is a brutally tough business!

  4. Hi Kevin - I have been using your Monte Carlo simulator for a few years now...good stuff! I was hoping you could give me some ideas for plotting out possible equity curves given a 60% win rate with profit target=stop loss at a fixed contract size and no compounding. Using Excel I have used the rand() function to scramble up a list of 100 theoretical trades and calculating a running net P/L to generate a possible equity curve, but this is slow and cumbersome. There must be a better way to simulate a large number of potential outcomes given these trading statistics and plot them on a single chart. Any suggestions? thanks for your time!

  5. Great to hear you are having fun with the Monte Carlo simulator! I use it almost everyday - it is one of the last "tollgates" I use to determine if a strategy is viable.

    Sounds like you are looking to produce a plot with say 10 or 20 or 100 different potential equity curves. If you e-mail me at kdavey@kjtradingsystems.com I can send you an example spreadsheet of that.

    You can also find online "Equity Monaco" by Tick Quest or NeoTick. It does that, and is free. It, too, is a good tool.