Is It Broken?

Note: Sorry in advance for the long post.  I hope you find it worth your reading time!



A reader posted this terrific comment: "I'm curious about your take on the performance so far. Is it a question of the system being misaligned with the modus operandi of the current market, bad money management, of something else?"

This is an outstanding question, because we've all been there - you are in the midst of trading a supposed "winning" system, yet losing money doing so!

This question is like an onion, with many layers.  So, let's start peeling...


First, the reader is asking because my contest system is losing money.  That losing can't be denied.  But notice how he asks the question - he is thinking/wondering something is wrong (misalignment, bad money management).  Makes total sense, of course.  If it isn't making money, something must be wrong, correct?

Well, maybe something is wrong, and maybe not.  As I write this, I honestly don't know the answer. We'll examine that together later.

But for now, that thinking brings us to the second layer on the onion.  Most people, myself included, usually assume something is wrong with a trading system when it loses money.  And once you convince yourself that something is wrong, the normal human response is to try to "fix" it.  

Change parameters, change instruments, change stops, add a filter etc. Just change SOMETHING!  Get it working right!

Usually, that is the wrong thing to do (unless the system is indeed "broken").  Making fixes as you go usually ends up like being on a hamster wheel - you go round and round, but never get anywhere.  You are chasing performance, and you end up making bad decisions because of it.  So the lesson here is to be aware of the questions you ask yourself - you may be assuming things which will lead you astray.

Why is trying to implement a fix wrong?  The answer is in the third layer of the onion - probabilities.  Many people don't realize that even a historically profitable system may have a prolonged losing streak in the future.  For any system, there is a chance - a non-zero probability - that you will lose most/all your money trading it.  

In the long run (years and years) the system might be profitable with 99% probability, but in the short run ANYTHING CAN HAPPEN.  And the shorter the time frame, the more likely it is that something bad will occur.  You have to give the probabilities enough time to kick in.  Think of a casino.  On any one spin of the roulette wheel, the casino may lose money.  But since the odds are in the house's favor, over many tables, and many spins, they (the casino) are almost guaranteed to win.


OK, so what does this actually mean?  Here is a real world example.  Unfortunately for me, I'll use my losing contest account.  This analysis will answer: "Is Kevin's contest system broken?"

For my contest system, I had previously determined through testing that on "average" (keyword here) should get something like this:

Avg Profit Per Trade = $206
# Trades per Year = 101
Avg Profit Per Year = $20,806

At this point, I should be up roughly $10,000.  Instead, I'm down around $2,000.

"Is this kind of actual performance possible, given these historical statistics?"

To answer the question, I'll use what is called Monte Carlo analysis.  Basically, you put all the trade results in a hat, and randomly pick one after another, until you've pulled 50 trades (in my case) out of the hat.  Adding these values up, you build an equity curve.

Obviously, if you do this one time, you might get lucky, pick all winning trades, and have a great equity curve.  Or, you might pull all losers, and have a terrible equity curve.  There is a low probability of each of these possible equity curves happening.  But, the point is ANYTHING CAN HAPPEN when you do this process once.

So, to get meaningful results, you repeat this process 100 or 1000 or 10,000 times.  Then, you can see some interesting things.

I built a Microsoft Excel spreadsheet to do this.  It is based on the simulator I offer free on my website.

Here are the results, with my system...


1.  Average Performance [Dark Gray line]  The chart below shows, on average, how my contest account should be doing.  After 6 months (50 trades), I should have about $10,000 in profit.  That agrees with the earlier analysis.  The line is a little wavy, due to the random nature of the simulation.

2.  Actual Performance [Blue line, with dots]  Here is my actual performance, overlaid on the same graph.  Yuk!  But, it is a great visual check of where I "should" be and where I am actually at.  Based on this, maybe there is a problem...

3.  95% Best Case Performance [Green Line]  If I run the simulation 100 times, some of the equity curves will look great.  This line is the 95% percentile line, meaning that there is only a 5% chance that the performance will be better than this.  If my performance was above this line, I'd be happy, but I'd also be wondering if something was "wrong."  Good "wrong," but still a concern.

4.  5% Worst Case Performance [Red Line]  Again, running the simulation multiple times gives you this curve.  It says there is a 5% chance that my performance would be below this line.  I'd be worried if it was below the line - something could be broken.

At this point, you might be thinking "All this analysis is ridiculous!  You still aren't making money!!!  What does all this goofy analysis tell you?!?"


Here is what I learned:

A.  My current performance is within the range of what is expected (above 5% line, below 95% line).

B.  My actual performance is at the 13% percentile. Meaning, I am currently pretty unlucky, and my antenna should be up that something may be wrong with the system.  Not panic time yet, though.  Not time to change the system.  But, I want to be above 50%, and until I get there, I'll be closely watching.

C.  If I want a "guaranteed" profit, I have to trade this system for at least 100 trades (imagine when the red 5% line crosses zero).  In other words, I have to be patient, and give the probabilities time to work.

D.  At this point, I am in the "major disappointment" zone, not in the "panic" zone.

I hope you found this long post useful.  It was useful to me!