How Should It Look?

I received some questions on my Monte Carlo analysis.  Basically, I use it to compare my actual performance to my expected performance (based on historical data).

In the last post, I found that my contest account is underperforming, and is close to the area where I might have to make some tough decisions/changes.

For reference, below is an example of another of my systems (Full disclosure: I sell this particular system on my website).  It is behaving pretty much as expected, although currently below the average line.  So, I wanted you to see what a "typical" performance chart looked like.

To answer the question, I create similar charts for every system I trade, although usually I just have the average line and actual line on the chart.  That way I can easily see how things are going!

Is It Broken?

Note: Sorry in advance for the long post.  I hope you find it worth your reading time!



A reader posted this terrific comment: "I'm curious about your take on the performance so far. Is it a question of the system being misaligned with the modus operandi of the current market, bad money management, of something else?"

This is an outstanding question, because we've all been there - you are in the midst of trading a supposed "winning" system, yet losing money doing so!

This question is like an onion, with many layers.  So, let's start peeling...


First, the reader is asking because my contest system is losing money.  That losing can't be denied.  But notice how he asks the question - he is thinking/wondering something is wrong (misalignment, bad money management).  Makes total sense, of course.  If it isn't making money, something must be wrong, correct?

Well, maybe something is wrong, and maybe not.  As I write this, I honestly don't know the answer. We'll examine that together later.

But for now, that thinking brings us to the second layer on the onion.  Most people, myself included, usually assume something is wrong with a trading system when it loses money.  And once you convince yourself that something is wrong, the normal human response is to try to "fix" it.  

Change parameters, change instruments, change stops, add a filter etc. Just change SOMETHING!  Get it working right!

Usually, that is the wrong thing to do (unless the system is indeed "broken").  Making fixes as you go usually ends up like being on a hamster wheel - you go round and round, but never get anywhere.  You are chasing performance, and you end up making bad decisions because of it.  So the lesson here is to be aware of the questions you ask yourself - you may be assuming things which will lead you astray.

Why is trying to implement a fix wrong?  The answer is in the third layer of the onion - probabilities.  Many people don't realize that even a historically profitable system may have a prolonged losing streak in the future.  For any system, there is a chance - a non-zero probability - that you will lose most/all your money trading it.  

In the long run (years and years) the system might be profitable with 99% probability, but in the short run ANYTHING CAN HAPPEN.  And the shorter the time frame, the more likely it is that something bad will occur.  You have to give the probabilities enough time to kick in.  Think of a casino.  On any one spin of the roulette wheel, the casino may lose money.  But since the odds are in the house's favor, over many tables, and many spins, they (the casino) are almost guaranteed to win.


OK, so what does this actually mean?  Here is a real world example.  Unfortunately for me, I'll use my losing contest account.  This analysis will answer: "Is Kevin's contest system broken?"

For my contest system, I had previously determined through testing that on "average" (keyword here) should get something like this:

Avg Profit Per Trade = $206
# Trades per Year = 101
Avg Profit Per Year = $20,806

At this point, I should be up roughly $10,000.  Instead, I'm down around $2,000.

"Is this kind of actual performance possible, given these historical statistics?"

To answer the question, I'll use what is called Monte Carlo analysis.  Basically, you put all the trade results in a hat, and randomly pick one after another, until you've pulled 50 trades (in my case) out of the hat.  Adding these values up, you build an equity curve.

Obviously, if you do this one time, you might get lucky, pick all winning trades, and have a great equity curve.  Or, you might pull all losers, and have a terrible equity curve.  There is a low probability of each of these possible equity curves happening.  But, the point is ANYTHING CAN HAPPEN when you do this process once.

So, to get meaningful results, you repeat this process 100 or 1000 or 10,000 times.  Then, you can see some interesting things.

I built a Microsoft Excel spreadsheet to do this.  It is based on the simulator I offer free on my website.

Here are the results, with my system...


1.  Average Performance [Dark Gray line]  The chart below shows, on average, how my contest account should be doing.  After 6 months (50 trades), I should have about $10,000 in profit.  That agrees with the earlier analysis.  The line is a little wavy, due to the random nature of the simulation.

2.  Actual Performance [Blue line, with dots]  Here is my actual performance, overlaid on the same graph.  Yuk!  But, it is a great visual check of where I "should" be and where I am actually at.  Based on this, maybe there is a problem...

3.  95% Best Case Performance [Green Line]  If I run the simulation 100 times, some of the equity curves will look great.  This line is the 95% percentile line, meaning that there is only a 5% chance that the performance will be better than this.  If my performance was above this line, I'd be happy, but I'd also be wondering if something was "wrong."  Good "wrong," but still a concern.

4.  5% Worst Case Performance [Red Line]  Again, running the simulation multiple times gives you this curve.  It says there is a 5% chance that my performance would be below this line.  I'd be worried if it was below the line - something could be broken.

At this point, you might be thinking "All this analysis is ridiculous!  You still aren't making money!!!  What does all this goofy analysis tell you?!?"


Here is what I learned:

A.  My current performance is within the range of what is expected (above 5% line, below 95% line).

B.  My actual performance is at the 13% percentile. Meaning, I am currently pretty unlucky, and my antenna should be up that something may be wrong with the system.  Not panic time yet, though.  Not time to change the system.  But, I want to be above 50%, and until I get there, I'll be closely watching.

C.  If I want a "guaranteed" profit, I have to trade this system for at least 100 trades (imagine when the red 5% line crosses zero).  In other words, I have to be patient, and give the probabilities time to work.

D.  At this point, I am in the "major disappointment" zone, not in the "panic" zone.

I hope you found this long post useful.  It was useful to me!

Forces Are Aligning...Against Me

Sometimes it seems like the market is just conspiring against me.  Who knows, maybe it really is - after all, the market's job is to transfer money.

The last few weeks have really felt that way for me.  It is confidence sapping, and emotionally depressing.

The only cure is to push through - continue working on new projects, writing, etc.  And of course, continue trading per the plan, without emotional decisions interfering.  So, continue to do everything as normal.  Experience, though, tells me that it is much more enjoyable and fun to do all these activities as equity is rising!

Still down about 20% as of Monday close.  But looks like more down on Tuesday...

Contest Position Update

Contest Position Update Information

I think there is a useful lesson in my performance for small traders, so I thought I'd share it:

Total Open Equity: +$1000 approx
Equity/Initial Margin = 100% (can't add new positions)

Big Losers: 2 (1 is near stop)
Little Losers: 3
Little Winners: 3
Big Winners: 2 (for both of these I have doubled size)

Since I can't add new positions, I have had to skip a bunch of signals.  Currently, there are 6 open positions I missed:


Big Losers: 1
Little Losers: 0
Little Winners: 3
Big Winners: 2



There is an important lesson here. The idea with system trading is that you take every signal, since the results of any one signal are essentially random (anything can happen).  If your system has an edge, then over time taking every signal is the only sensible way to go.

If you are undercapitalized, like I am in this system account, then you may have to skip signals.  If earlier trades had done a bit better, I'd have enough equity to not skip signals.

To repeat an earlier post, I funded this account with the bare minimum, since the contest is based on % return.  If the contest was instead $ return, I would have initially funded it with more money.  Again, decisions like this go back to your objectives and goals.

LESSONS:


1.  FUND YOUR ACCOUNT WITH ENOUGH MONEY TO TAKE ALL TRADES


2.  MAKE SURE YOUR FUNDING AND SYSTEM ARE IN AGREEMENT WITH YOUR GOALS AND OBJECTIVES

Weekend Update, Step 10

Contest Update

Almost 6 months into the contest, and I'm down 20%.  Not exactly how I had envisioned!  But then again, trading has never been a smooth upward ride for me (and, I suspect, is not that way for most people).

Flat/down periods, followed by quick jumps up, is generally how things go, at least historically.  Of course, this can be mentally draining, since the flat/down periods tend to make you think you are trading incorrectly/poorly.  The markets have a way of breeding indecision, lack of self confidence, impatience, frustration, worry, and a million other bad feelings.  It is quite amazing, actually, at how the market can ruin your psyche.

At this point, I'll continue to ignore those bad feelings -  I'll keep sticking to the Trading Plan, and see where it leads!


************************************


The Trading Process - Incubate - Step 10

In this post, I'll give you a tip that is one of the best things I do.  It is also one of the hardest.  It is called "incubation."

The idea is that once you develop a strategy, following all the correct steps, you don't trade it right away.  You just put the strategy aside for 3-6 months.

Of course, since you just finished the development, you want to start trading it TODAY!  Just think of all those dollars headed out the window, as you let the strategy sit!  That is what makes this step so difficult - no one wants to spend months developing a strategy, only to let it sit for months on end.

BUT, how do you know for sure that you didn't "cheat" while developing the strategy?  How do you know that you used the correct process?

The simple fact is nothing (except for real money trading) beats results with future data, and that's what you get by incubating.

I personally put a strategy aside, and look at it once a month.  If, after 4-6 months, the performance is close to my historical testing, then I may allocate money to it.

Many times, especially when I first started using the incubation process, I weeded out many "bad" strategies.  In some cases, I went back and determined that my strategy development process had some flaws.  Thankfully, incubation hleped me make improvements, without costing me money!

Next: If there is a fire, you better know where the exit is!

Entries and Exits

There is a good discussion going on over at the Tradestation User forum about entries and exits.  If you are a registered TS user, you can access it here: https://community.tradestation.com/Discussions/Topic.aspx?Topic_ID=120700

For everyone else, I thought I'd post a couple of comments I left today about entries and exits...


"As for me, my history shows that I am pretty bad at getting "good" entries, and pretty bad at getting "good" exits. For almost every trade, I seem to enter at the wrong time, and exit at the wrong time. New trades almost always go against me - usually immediately. Existing trades sometimes go from profit into loss.

My systems sometimes seem to have a penchant for buying at the peak, and selling at the low (some of my favorites I post at my blog). Through all that I seem to survive. For me, then, I think it is when I exit - before too much damage is done on losses, and trying to ride winners for a long time. It is tough, though.

To me, what I said about having "bad" entries and having "bad" exits is the funny part about trading. You can do well overall, even though it seems like you aren't doing well on most trades.

Maybe it is me maturing as a trader. In the old days, I used to really get bummed out when a new trade would immediately turn against me. Now I realize that almost every trade - eventual winners and eventual losers - are losing trades at some point. Most are winning, too at some point. So, I try to just let things play out, per the system, realizing my results will be far from "perfect."

The other thing I have found is that if I start to actually deep down believe I am a great trader, then that is when I am most vulnerable to making big mistakes (changing a system with improper testing, adding size too quickly, abandoning research because everything I have is doing great, etc).

So, I try to treat every day as my last as a trader - before I have to go back and get a "real" job."

Underwater, Tapped Out

Well, I am about 18% down from the start, at almost a 50% maximum drawdown from my peak equity.

To make matters worse, the trades I am currently in have gobbled up all my initial margin, so I cannot put on any more trades.  Of course, the few trades I've missed already have cost me probably a couple of thousand dollars.

Both of these facts are a result of starting out with a small account.  There is a good lesson here for people with small accounts - realize your drawdown will likely be more severe, and you might have to skip good signals.

But, I'll continue to forge on.  Hopefully things will turn up soon.

Step 9 - How To Backtest

Contest Update:

Still underwater (about 10% loss from start).  Still trading exactly as planned (except where initial margin becomes issue).  Still VERY disappointed at the last month of performance.  Still feeling more than a bit depressed and embarrassed at having to show this to the world.

I hope that readers get one takeaway from my current performance: anything can happen in trading.  Historically winning strategies can turn negative (for either a little while, or maybe forever).  Don't position size assuming that you will be profitable - that is trading suicide.


**************************

The Trading Process - Backtest the Right Way - Step 09

Last time we discussed "expectancy."  If your trading approach doesn't have a positive expectancy, DON'T TRADE!  It is critical.

Of course, to get historical trades to determine your trading system expectancy, you must backtest.

You can either backtest by hand, or use trading software to help you.  Backtesting can be very tricky, though.  Here are a few tips to help you backtest properly:

1) Make sure you include realistic costs for commissions and slippage.  I typically use $25-50 per contract per round turn trade.  My numbers are based on actual trading.  Many strategies look great with $0 commissions and/or slippage.  THESE SYSTEMS ARE FANTASY!

2) If you use backtest software, learn the tricks and pitfalls in it.  In Tradestation, for example, the default setting for limit orders will assume a limit order is filled if touched.  Traders know that in most instruments, you'll never get filled on a "touch" of your limit price.

3) Make sure you aren't "peeking" into the future.  This is really easy to do if you backtest by hand or with Excel.  An example of this would be buying today's close if tomorrow's open is up - you can;t know this in advance.

4) If your results look too good to be true, they probably are.  You either done something wrong, optimized your backtest too much, or curve fit rules to match the data.

5) Your backtest should definitely include either a significant out of sample test period, or should use walkforward backtesting.  If you don't know why this is important, make sure you research it and learn.  I guarantee that following this step alone will save you thousands.  Only the inexperienced optimize over all their data, and then start trading.

Obviously, proper backtesting is a huge topic.  I've just scratched the surface.

Next: Incubation, not just for baby chicks any more!

The Bug and The Windshield

In trading, sometimes you are like a car windshield.  As you roll down the road, anything in your path gets out of the way, or hits you and gets smashed.  But you just keep going.

Other times in trading, you are a bug, just merrily flying along, when out of nowhere this big windshield appears, and splat! you get crushed.

I always like to say: "In trading, sometimes you are the bug, and sometimes you are the windshield."

Right now, I feel like the bug!

Everything is Relative

If I told a non-reader of this blog that I was up 19% for the year, they might be duly impressed.

But, readers of this blog realize that 19% isn't so great, considering I was once up over 60%.

A straight line equity curve, ending in 19% up, is certainly more preferable to what my curve has done.

So, performance is relative.  Keep that in mind as you search for suitable trading strategies.  The ups and downs (especially the downs) is what will kill your resolve, shatter your confidence, and cause you to abandon ship right before the equity curve shoots up.

Not that I complaining (although I am disappointed somewhat).  I just wanted to put it in perspective.

Positive Expectancy

Contest Update:

Up about 11% for the year (see chart below).  All I can do is follow the plan.  I say that over and over, but it is the best thing I can possibly do. Most people fail when they start to deviate from their plan.  This contest system may still fail, but I know it has historical positive expectancy.  That means it has been profitable in the past.  But, I need to follow the plan to reap the benefits of positive expectancy.


That is a good intro to step 8 in my continuing series of "Developing a Trading System"...


Comments and questions, as always, are encouraged...




Trading Process, Step 08 - Have Positive Expectancy



Many people look at a chart, "see" a few instances of a profitable pattern, and then start trading it.


Other people see a flashy new indicator being sold by someone who probably doesn't even trade.  Every example shown leads to profit.  It is the Holy Grail indicator!


Does either case sound familiar?  Unfortunately, that is what most people do before they trade - they find a few profitable examples that back up their thoughts (and conveniently ignore or hide the losing examples), and then start trading.  That is the wrong way to do it.


The right way to evaluate a strategy, whether you are backtesting or evaluating in real time, is to:


1) Have a statsitcially significant number of trades.  5 or 10 trades is not enough.  You want at least 50, preferably 100's or more.  In my contest account, I had over 500 trades evaluated as part of my research.  In my SFE Trading system (available on my website), I evaluated over 3,000 trades.


2) Have an evaluation period that includes bull markets, bear markets, flat markets.  A month or two of market action is never enough.  You really need years.


3) HAVE A POSITIVE EXPECTANCY SYSTEM.  Here is how to calculate it:
Expectancy = ((Probability of Win * Average Win) – (Probability of Loss * Average Loss))/(-Average Loss)
This number must be positive, ideally 0.2 or higher.  The higher the better, although if it is too high, I'd wonder about overfitting or  over optimizing, or some other backtest error.


If you have all 3 of these items, your chances of success go up a great deal.


Next: How To Backtest

The Market is Messing With Me!

Have you ever had that feeling - one that the market is just playing around with you, kind of the way a cat plays with a mouse (just before the cat eats the mouse)?

That is how I feel now.  So, the market turned against my positions in the past few weeks, and I could not enter 3 positions in the past few days.  Not enough margin in the account to open new positions.

So, take a guess: do you think these positions, ones the market would not let me enter, were profitable, or not?

If you said profitable, you were right.  $1,500 in missed contest account profit.  Ugh.

[But in the end, I won - I opened those positions in another account (non-contest) I have.  Of course, I'd rather have that $1,500 in my contest account!]

Contest Update:

Up about 15% for the year.  Not feeling very confident about this endeavor.  But that is OK - my emotions, personal feelings and state of mind don't really matter - as long as I FOLLOW MY PLAN.

Embarrassing

This is not going to plan!  It is funny - it is one think to think beforehand "Hey, I could have a 50% drawdown," but it is a TOTALLY different thing to have to live it, in real time (not accelerated backtest time).

This is starting to get ugly!

Still Following The Rules...

Up 5.5% for the year.

Like always, I'm still following the rules exactly.  Sometimes it works in my favor, sometimes not.

In times like this (big drawdown period), it is important to remember that anything can happen.  You can do everything right when developing a great system, you can have perfect psychology, discipline and mental control, you can execute your plan flawlessly, and yet YOU CAN STILL LOSE!

Why? - Trading is all about probabilities.  People who forget that will lose in the long run.

Excellent Question From a Reader

A reader asks:

"Does your in-sample testing account for almost a 50% drawdown? If not at what point do you say the strategy is not working any longer?

Thank you for your blog and your transparency. It's encouraging."


Thanks for the question.  The transparency unfortunately sometimes makes me look like a fool.  Seriously, what kind of "professional" trader has a 49% drawdown, and a 41% drawdown?  Having to post my equity graph the past week has been awful - really, it is 10 times worse than the $5,000 I've lost during the drawdown.  It is an ego deflater (which is probably a good thing, in the long run - I know arrogant traders, and I know rich traders, but I do not know any rich, arrogant traders.).  


Many people would have quit after that first big drawdown.  Emotionally, I can relate.  BUT, if the numbers predict it could reasonable happen, then it doesn't make sense to quit.  That is the case here.


Let's look at the system I am trading.  I started with only $10,000, which I knew would likely put me in a precarious drawdown situation, sooner or later.  But I was going for absolute percentage return, so it makes sense to start with the least amount of capital you can.


There is an important lesson here, and it all goes back to what your goals and objectives are.  My goal was percentage return.  If the goal was risk adjusted return, I would have done things differently.  Started with more money (probably not a help), or likely traded a whole different system (one that had more limited upside, but less drawdown potential).  


Anyhow, I ran the contest system through my Monte Carlo spreadsheet (free, on my website), and I determined that within a year, there was a 65% chance that I would have a maximum drawdown of at least 50%.  In other words, there was a very good chance of a 50%+ drawdown.  The graph is below.


The flip side, is how much gain could I get out of this system?  The same analysis shows I have a 60% chance of at least tripling my money.  And that assumes that I keep trading 1 lots through the year.  I'd probably increase size, so I might do even better.


Basically, I decided that these were odds I could live with.  Not everyone can.  And I certainly would not have these types of odds for my other accounts.


For this system, since it is a contest, I am going to quit only if I cannot place a trade.  Again, in my "normal" accounts I'd never do that, and I don't recommend you do this.  But this approach fits my objectives, which is the key.


If this analysis in confusing, please feel free to ask questions.  I want to make sure readers understand.

"You're Going The Wrong Way!!!"

Why do I feel like I am riding in the car with these guys - Steve Martin and the late John Candy?  Along with my contest trading system?

http://www.youtube.com/watch?v=_akwHYMdbsM


The ironic thing is May was a terrible month for my contest account, but overall (multiple accounts, 11 total systems) was my best percentage gain month so far in 2012 (over 13%).  Yes, it has been a tough year - last year, I had 5 months over +13%.


Maybe June will have everything go the right way!