The Process -- Techniques for Measuring Trade Signal Aptitude
I don't know if it's the best way. I don't even know if it's the right way. I do know that it's my way, and that my way is the way of the student.
Disclaimer: the following post is an organized representation of my research and project notes. It doesn’t represent any type of advice, financial or otherwise. Its purpose is to be informative and educational. Backtest results are based on historical data, not real-time data. There is no guarantee that these hypothetical results will continue in the future. Day trading is extremely risky, and I do not suggest running any of these strategies live.
I'm working on Strategy 14 (S&P 500 Mean Reversion) and I realized that there are a few simple techniques that I use when I am working on strategy design that I haven't really talked about. Let me rephrase that, I've talked about them before, during the indicator testing series or when presenting a futures strategy, but I haven't really gone into more detail about what I mean.
I've seen this pattern a lot in publications that discuss trading strategies. There is a broad overlook and a discussion of concepts, but never really a discussion on what it looks like to put some of these concepts into practice. I provide strategies, indicators, research, and code, but I realized I don't really discuss how to use some of the tools that are discussed in my articles. So, this is the first article in a series I will label "The Process", an attempt to journal about my research and design process.
I want to note that the actual tools I am using should not make much of a difference when it comes to the concepts I am talking about. Even though I am using RealTest here, you should be able to reproduce the same simple logic in any well constructed backtesting engine.
Also, this is just my current process. I am always willing to adapt if necessary, so it may not be my process forever.
The Tools
I only have a few tools that I currently am using for trade system research. For backtesting, I use RealTest. For just about everything else, I use Python and build my own tools. I am a bit of a software/gear/tool whore, so I spend a lot of time looking at different features of products and weighing the benefits. It's a problem. However, it does mean I spend a lot of time trying to make sure I choose a tool that isn't going to fail me and aligns with my values.
That last part if difficult. In this world of tech, so many products either don't quite do exactly what I want or I don't like their business model or whatever. I also hate being stuck in a single ecosystem. My values and needs won't be the same as yours (or maybe they are, who really knows?), so I would do my own research to find tools that fit your mission. Here is a list of a few things that lead me using RealTest as my backtesting engine:
It's (well) written in C. Even if it gets updated, I am confident that the parts that matter will still be written in C. It's very efficient with large amounts of data.
It uses standard csv for data, so you don't have to subscribe to any one data source. You just need to make sure that your data is in a friendly format (hello, Python).
It also has direct import capability with yfinance, Tiingo, and Norgate Data. This may or may not change in the future, but it isn't a deal breaker for me. I can use Python to create csv data.
Simple scripting language that is very similar to a functional programming language or spreadsheet function formulas.
It can test multiple strategies and across multiple instruments, making it very good for portfolio strategy development.
It has a very active community forum and a very active and responsive developer who is also a trader.
The fact that it can used a standard csv file for importing any data you want is awesome. It would be cooler if it could access databases, or import csv data with custom headers, but maybe in the future. There is no promise that I stay with RealTest as my backtesting engine. If something better comes along, I'll have a look at it. At the moment, it is the best tool for my mission.
I use Norgate Data as my data source. I do this because they have both direct import capability for RealTest and a good Python SDK. That means I can test my tools and indicators in Python using the same data I intend to backtest with. Starting to see the pattern forming here?
I also use Sierra Chart, but currently I only use it for trading manually. I chose it for many of the same reasons as I did RealTest. Since I trade futures, I can actually export data from Sierra Chart, convert it to the proper format for RealTest, and then test futures strategies with the data that I would actually use for trading. Sierra Chart is also meant to be the platform that I use to run live futures trading strategies. However, that isn't in the immediate future, so for now it is just my trading platform.
Lastly, I use Python (who saw that coming). This could be any coding language, really. Python has a lot of libraries and tools out there that really help with data. Of course, I am doing my best to use very few of them, so I don't get that advantage. There are many others, but they don't matter. The point is that having knowledge of a programming language is a critical part in how I research. It is the middle ground where I create any specialized tools that I need and manipulate data.
The Workflow
Now that there are 7 paragraphs discussing the three tools that I use for my process, let's talk about workflow. Don't worry, this section will be brief. I create and research indicators/features in Python using the same data that intend to test with and preferably even trade with. That last part is more important with futures trading (and maybe intraday). Then, I take my findings and I start testing it in RealTest.
That's it.
Testing an Idea
Now, we are finally at the topic of discussion for this post. Measuring a trade signal's aptitude.
If you go back through my posts, you will find one that discusses building a indicator threshold optimization tool that builds a table of threshold values with associated profit factors and a single "best" indicator value for long and short positions. It measures this profit factor by each bar. That means, instead of holding a trade for x amount of bars while indicator is above/below x value, we calculate each bar as a trade. On top of that, it calculates from close-to-close to account for overnight noise.
Well, this can be done in RealTest too. In fact, it is something I do when I adapt an indicator from Python into RealTest Script, regardless of what the test results in Python are. I know I could export some values and run a comparison to results in Python, but that's a lot of work just to get an indicator to exactly mimic one written in a full programming language. Instead, I adapt it, then I eyeball the charts and make sure that they appear similarly sound.
Simple enough, right?
After that, I create two separate Strategies for the indicator: one long, one short. Each of these strategies are set to trade a single signal, one instrument, and with a simple ATR (average true range) and equity based quantity sizing formula.
Those Strategies look like this: