Automated Trade Strategy 44X hits a roadblock--I think I broke the NinjaTrader backtest engine.

This could be the first non-user error I have encountered or a user-induced black swan event. I don’t know how to fix it yet, and I don’t know if I will spend much time trying.

Disclaimer: the following post is an organized representation of my research and project notes. It doesn’t represent any type of advice, financial or otherwise. Its purpose is to be informative and educational. Backtest results are based on historical data, not real-time data. There is no guarantee that these hypothetical results will continue in the future. Day trading is extremely risky, and I do not suggest running any of these strategies live.

I was excited to work on our first strategy (Automated Strategy 44X) this week. Instead of offering a variation of a more recent strategy, I decided to convert 44X over to the new framework. It had good results in its original backtest, so I thought this would be a good one to convert for the community. I also decided to challenge myself. The purpose of this framework was to make strategy creation fast. The objective is to code 44X into a custom signal class and implement it into a strategy in under three songs.

I chose the songs I wanted to listen to, closed all my windows (no cheating), hit play, and got after it.

First up, “Workout Plan” by Kanye West. Alt+2 to open the terminal, tmux attach to my HGT code base session, open vim, navigate to the signal folder, and create a new Signal44.cs file and open it. Harpoon that shit, go back and harpoon the latest strategy (ATS24), the original 44X, and then switch back to the new signal file. Start coding. We are blazing.

Next up, “Gangnam Style” by PSY. Oh yeah, I was feeling it. By this time, I had the base code for the signal and almost all of the logic. During this process, I decided to leave out some of the extra criteria for the strategy. I am going to start with the underlying logic and add to it during the testing phase to see if the additional criteria are necessary. I finished the signal code about halfway through the song.

Finally, “Tuscan Leather” by Drake. If you know this song, you know it’s kind of long. I backed out to the root directory and created the ATS44.cs file. I yanked the code from ATS24, pasted it into the new file, and changed the name and the signal variable to 44 instead of 24. That’s it. I gloated a bit, then sat back and enjoyed the last 5 minutes of the song.

Jackpot.

I went to the fridge and grabbed a beer for the testing phase. I had no compile errors to take care of, and everything looked great. I set the backtest dates, chose the new strategy, cracked open the beer, took a long swig, and ran the backtest.

If you have access to the GitHub, and if you have looked at some of the newer code, you will see a bunch of random print statements inside some of the functions. These are remnants of debugging those specific functions. They are placed throughout the process of generating trade signals and executing trades so I can track the process in the output windows. This way, I can see how it runs through the process and ensure that my logic functions as I intended.

The backtest started, and everything started printing to the output window exactly as it should. Zero errors. I was fucking ecstatic. I just coded this entire micro-project in about 10 minutes, and it worked without throwing a single error. I’m improving.

But wait, the results look a little funky. Why are there so few trades? The results are similar to what we got the first go around (without the extra criteria, hmm), but it only made 20-something trades.

That’s odd, I thought. Let’s try some more beer and run it again. I made sure the dates went back at least a year, but I got the same results. I finished the beer and got another one. I guess it was too good to be true. Time to debug the code.

The first thing to do is go through the output window and make sure that everything is firing the way it should. There are about a week’s worth of trades in the backtest results that didn’t have problems. The last trade it fired before stopping is where I will find the issue. Except, there isn’t a last trade. The output window shows that the last trade was initiated, but then… nothing. That’s weird. Here are some screenshots.

In the first image, you can see that I highlighted the last trade (in the output windows) signal for reference. The next image is more information from the output window to see how the previous trades functioned and the order of print statements. We are following every step that happens after a trade signal is generated.

Why did it stop trading right there?

Let’s change the dates and see what happens. The trades stop on 3/2/2023 in this test. Let’s change the date to 3/1/2023 and see what happens.

Interesting. It broke on the same trade. What happens if we start it on the day after that trade?

It worked, but not for long. It fired more trades before breaking this time, but it still failed. This time, it failed on a short, so we can rule out something wrong with the long-side trades.

At this point, I was annoyed. I was down two beers and didn’t have a solution yet. Everything is wrapped in try/catch methods to catch and print errors, but there are none. I have print statements inside every process, even the OnOrderUpdate() function, to ensure there isn’t an error. There are no errors being thrown, period. At least, none that I can figure out or see.

So, what is going on?

I have no fucking clue. I crushed two more beers working on it before I decided to walk away from it for the night. I returned with a fresh look this morning, but I’ve still got nothing. There are still no errors, nothing out of place in the logic that I can see, and we still get random failures. The interesting part is that it keeps failing on the same trades. Shift the dates to after that trade, and you get the next set of results until it fails again.

Is this a user error or a NinjaTrader error?

Did I manage to break the backtest engine in a way that I can’t even diagnose the problem?

The only thing I know for sure is that I do not understand what is happening here and don’t care. This is not the first issue I have had with NinjaTrader, but it is the first one I haven’t been able to diagnose. This just supports my decision to move away from NinjaTrader to research automated trade systems. I’m well aware that something is going on here that I don’t understand. I just don’t have the desire to fix it since I don’t intend to stay with this platform. It has been a great learning experience, but it’s time to say goodbye to NinjaTrader for now.

Strategy 44X had good backtest results when I originally posted the article, and that has stayed the same, even though we can’t get the full picture in backtest results. Here are some examples of what they look like for the trades it executes. You can look at the dates in the photos for reference, but I randomly chose the start dates. Also, these are for the strategy without any extra criteria. Those with access to the GitHub can see exactly which criteria was left out from the original strategy.

The results that we do get look promising (especially the accuracy) and similar to what we got before when we first turned this strategy into a custom NinjaScript strategy. The problem is that we can’t trust this strategy to run live as it is.

Where does that leave us?

In the same place as before. I already had intentions of migrating away from NinjaTrader. It is just happening sooner rather than later. The only difference is that I couldn’t deliver a working strategy this week like I had planned. Despite that fact, I believe this is valuable information.

In the months I have been working with NinjaTrader, this is the first time I have encountered a problem like this. It has always generated an error message. Even when the logic is wrong, and a trade initiates and never closes, the strategy analyzer has always closed out an open position before delivering results. Here, we can see that it creates the entry order object. The problem is that it doesn’t do anything after that. It just stops, as if that execution never happened. Very strange.

Over the weekend, I am going to test out a Linux distro on my laptop and see if I can run it on my current laptop or if I need to buy a new one. If you are a nerd like me and want to know what distro I plan on using, hit me up in a DM, and we can talk some shop.

Next week, regardless of how my Linux experiment goes, I will dive into the QuantConnect documentation and start working on our strategies there. The first step will be porting our current strategies over to this new framework and testing them with the new engine. This will be fun. We will see if there are any significant differences between the results on these two platforms.

After that, anything goes. I have several ideas for strategies that I want to try out, and I am excited to be working on something that I came up with on my own. The first couple of strategies I want to research will be simple, like the ones we have covered so far, but I intend to address some issues with how these strategies function. For starters, I want to ensure there are very few optimizable variables and make those we use often (SL/TP) as dynamic as possible.

Thank you for reading, and I apologize for not being able to provide a working strategy this week.

Substack just launched the Direct Message feature for the chat. Feel free to reach out to me via DM if you want to chat.

Disclaimer: the following post is an organized representation of my research and project notes. It doesn’t represent any type of advice, financial or otherwise. Its purpose is to be informative and educational. Backtest results are based on historical data, not real-time data. There is no guarantee that these hypothetical results will continue in the future. Day trading is extremely risky, and I do not suggest running any of these strategies live.

The code for strategies and the custom functions/framework I use for strategy development in NinjaScript can be found at the Hunt Gather Trade GitHub. This code repository will house all code related to articles and strategy development. If there are any paid subscribers without access, please contact me via e-mail. I do my best to invite members quickly after they subscribe. That being said, please try and make sure the e-mail you use with Substack is the e-mail associated with GitHub. It is difficult to track members otherwise.

Feel free to comment below or e-mail me if you need help with anything, wish to criticize, or have thoughts on improvements. Paid subscribers can access this code and more at the private HGT GitHub repo. As always, this newsletter represents refined versions of my research notes. That means these notes are plastic. There could be mistakes or better ways to accomplish what I am trying to do. Nothing is perfect, and I always look for ways to improve my techniques.