Backtesting is a process of validating your trading strategy against historical data to see if it really works. It is one of the most important aspects of profitable trading system development. Through backtesting, a trader can simulate his strategy on past data instead of applying it on future unknowns and conclude if the strategy is accurate or not.
For example: Suppose you carefully observe a price chart for 1 month and discover that whenever the price crosses 20 days moving average, it continues to rise for a few more days. Now, you have two options:
- You can apply this strategy on live market and see if it works or not. That means you’ll place a buy order whenever price crosses 20 days moving average. TOO RISKY
- You can test the same strategy on years of historical data and be more confident about its performance.
The latter is termed as ‘Backtesting’ and is the industry standard method to gauge any trading system.
Process of Backtesting
In general, backtesting is a 4 step process:
Step 1: Formulate your trading strategy and its rules. If you are using an automated backtesting software, you may have to code your strategy too.
Step 2: Choose the financial instrument(s) on which you want to backtest your strategy on.
Step 3: Collect the historical price data for backtesting the strategy. The data should be accurate and cover a variety of market conditions. For swing trading strategy, atleast 5 years of data is recommended.
Step 4: Run your strategy rules on historical data to see if it works or not. Calculate important parameters like annual gross return, drawdowns and risk-reward ratio to gauge the performance of your system.
Step 1 to 3 above can be done manually, but step 4 can be cumbersome and time taking. This is where you may have to use automated software to perform backtesting.
See below, the backtest result summary generated by Amibroker:
The screenshot shows all important parameters to validate the performance of your system. If you calculate all these parameters manually, it may take months, but in Amibroker it gets generated in seconds.
Fine Tune your Backtesting results
The next obvious question is whether you can conclude on the performance of the system if the results look positive on historical data?
Well, 80% of systems perform well on the live market if the backtest results are good, but the rest 20% suffer from something called Optimization Bias or Look-ahead Bias. But the good news is that there are ways to overcome these.
Since the trading system is tested on historical data with pre-defined variables (ex- moving average length), there is a possibility that the system may be overfitting the data. In other words, it performs incredibly well on the particular dataset, however, if the data or variables are tweaked a little it fails.
As markets are random, it’s imperative to eliminate overfitting in your trading systems. We cannot expect the future price movements exactly the same as the historical price movements (although Dow theory states the same).
There are few proven methods to get rid of overfitting issues and thus develop a robust trading system:
- Use the minimum number of rules or variables (indicators) in your trading system. You may get tempted to improve the accuracy of your system by using numerous indicators in the buy/sell rules, but this is a trap. The more the number of indicators, the more it gets overfitted on the historical data.
- Add randomness to the data or rules, and re-test the system. Perform this step iteratively multiple times and note the results after each iteration. Suppose you do this 100 times, and the system performs well at least 90% of times, then congratulations your system is ready to go live. This process is called Monte Carlo Simulation.
- Divide your historical data into two sets – In sample and Out of sample. Initial backtesting and optimization should be performed on in-sample data, while it should be validated on out-of-sample data. If the results look profitable in both the data sets, then the system is considered to be trustworthy. This process is iterated over multiple chunks of data to arrive at the best parameters. This process is called Walk Forward Optimization.
No 2 and 3 are very important for profitable system development, but if done manually they may take a lot of time. Amibroker software has a way to automate both of these.
See the sample Monte Carlo simulation results from Amibroker:
The first column shows the percentile level (the value below which a given percentage of test observations (realizations) fall). So say 5th percentile tells us that 5% of time observed value is below the shown amount. For example, the annual return value at 5th percentile (in this case 3.16%) means that 5% of tests (realizations) had annual profit less or equal than shown (3.16%).
Simple backtesting against historical data is ancient now. In order to excel in this competitive trading environment, its important that you adapt your system to the worst of market conditions. And this is when Monte Carlo Simulation and Walk Forward Optimization comes handy.