Walk Forward Optimization and Testing – Amibroker Tutorial

All amateur or professional trading system developers would be aware of Backtesting and Optimization concepts. In-fact, it is the most integral part of Algorithmic system development. However, what most people are not aware of is how to interpret the backtesting and optimization results. This leads to a trap and people lose enough money even when the trading system was proven profitable during backtesting. Walk Forward Optimization is an important method to determine the robustness or credibility of your trading system. Like simple Optimization, it is also a process to determine the best parameters for your trading system. However, in Walk Forward Optimization the system is optimized for a particular period of data to find the best parameters, and the obtained parameters are tested on the data following that period (known as forward data). If the parameters look profitable in both the data sets, then the system is considered to be trustworthy. This process is iterated over multiple chunks of data to arrive at the best parameters. The basic purpose of Walk Forward Optimization is to avoid curve fitting in simple optimization.

Check our article on Algorithmic Trading system development below:

Build your own Algorithmic Trading System: Step by Step Tutorial

Before doing backtesting or optimization, one needs to set up the data required which is the historical data of a specific time period. This historical data segment is divided into the following two types:

  • In-Sample Data: It is a past segment of market data (historical data) reserved for testing purposes. This data is used for the initial testing and any optimization and is the original parameters of a system under test.
  • Out-of-Sample Data: It is the reserved data set (historical data) which is not a part of the in-sample data. It is important as this ensures that the system is tested on another period of historical data not earlier thus removing any bias or influences in the checking of the system’s performance.

The process is to first develop a trading system using in-sample data and then apply the out-of-sample data to the system. The results of both cases can then be compared and tested. The entire process of Wak Forward Optimization can be better understood using the below graphic:

Walk Forward

Graphic Source: https://www.amibroker.com/guide/h_walkforward.html

Performing walk forward optimization manually would definitely be a tedious and time taking process. In order to make it easier, Amibroker has a inbuilt functionality for this.In the next section, we would go through the step by step process to perform Walk Forward Optimization.

Walk Forward Optimization in Amibroker

Let’s consider a simple EMA Crossover strategy where EMA Periods need to be optimized for Daily timeframe. Below is the AFL code for this strategy:

//------------------------------------------------------
//
//  Formula Name:    Walk Forward Testing
//  Author/Uploader: Trading Tuitions
//  E-mail:          support@tradingtuitions.com
//  Website:         www.tradingtuitions.com
//------------------------------------------------------

_SECTION_BEGIN("Walk Forward Testing");

SetChartOptions(0,chartShowArrows|chartShowDates);
_N(Title = StrFormat("{{NAME}} - {{INTERVAL}} {{DATE}} Open %g, Hi %g, Lo %g, Close %g (%.1f%%) {{VALUES}}", O, H, L, C ));

//Initial Parameters

SetTradeDelays( 1, 1, 1, 1 );
SetOption( "InitialEquity", 200000);
SetOption("FuturesMode" ,True);
SetOption("MinShares",1);
SetOption("CommissionMode",2);
SetOption("CommissionAmount",100);
SetOption("AccountMargin",10);
SetOption("RefreshWhenCompleted",True);
SetPositionSize(150,spsShares);
SetOption( "AllowPositionShrinking", True );
BuyPrice=Open;
SellPrice=Open;
ShortPrice=Open;
CoverPrice=Open;

//Parameters

MALength1 = Optimize("MALength1",20,5,200,5);
MALength2 = Optimize("MALength2",50,5,200,5);

//Buy-Sell Logic

Buy = Cross(ema( C, MALength1 ),ema( C, MALength2 ));
Sell =Cross( ema( C, MALength2 ), ema( C, MALength1 )) ;

Buy = ExRem( Buy, Sell );
Sell = ExRem( Sell, Buy );
Short = Sell;
Cover = Buy;


_SECTION_END();

As you can observe, the default periods for slower EMA is 50, and that of faster EMA is 20. This system would be optimized for NSE:Nifty for the time span of around 16 years (From 2000 to 2016). Other required parameters for this trading system has been mentioned in the above AFL code.In the AFL, we have used a function called ‘Optimize‘. This function instructs Amibroker that the variable needs to be optimized. Below is the signature of this function:

optimize( “Description“, default, min, max, step );

where,

Description: Parameter description that we are optimizing

Default: Default Value of the parameter to be shown if plotted in chart

Min: minimum value of the variable being optimized

Max: maximum value of the variable being optimized

Step: An interval used for increasing the value from min to max

For Walk Forward Optimization, click on the backtester settings and then click on ‘Walk Forward’. You would see the below window:

Walk Forward Optimization Settings

The settings in this window would be defaulted based on your data. So you actually don’t need to change anything, but still you have the liberty to do so.

  • Start and End dates mark initial period begin /end. This period will be moved forward by Step until the End reaches the Last date.
  • The Start date can move forward by step too, or can be anchored (constant) if Anchored check is on.
  • If you mark Use today then Last date entered will be ignored and TODAY (current date) will be used instead.
  • By default an “EASY MODE” is selected which simplifies the process of setting up WF parameters. It assumes that Out-of-sample segment immediately follows in-sample segment and the length of out-of-sample segment equals to the walk-forward step
  • You should use Easy mode (EOD) when testing on end-of-day data or Easy mode (Intraday) when testing on intraday data. The difference is that in EOD mode the END date of previous period and START date of next period are the same – thus avoiding gap between periods. Intraday mode set START date of the next period as NEXT DAY after END of previous period. That guarantees that boundary day is not counted twice when testing on intraday data.
  • In the Advanced mode, the user has complete control over all values, to the extent that they may not constitute valid WF procedure.
  • The “Optimization target” field defines the optimization raport COLUMN NAME that will be used for sorting results and finding the BEST one. Any built-in column can be used (as appears in the optimization output), or you can use any custom metric that you define in custom backtester. The default is CAR/MDD, you can however select any other built-in metric from the combo.

Once you defined Walk-Forward settings, please go to Automatic Analysis and press the dropdown ARROW on the Optimize button and select “Walk Forward Optimization” .This will run sequence of optimizations and backtest and the results will be displayed in the “Walk Forward” document that is open in the main application frame. The optimization process would run for few minutes or hours depending on the volume of data and number of parameters.

Below is the sample result of Walk Forward Optimization:

Walk Forward Optimization Result

How to interpret Walk Forward Optimization results?

Walk Forward Optimization results can be very useful to evaluate the reliability of your trading system. If a particular set of parameters works good in all the in-sample and out of sample data sets, then it indicates a fair possibility that the system would work in real market conditions.

Walk forward test determines the optimized system performance as follows:

  • Was it realistic? It is considered realistic if it could fit to the entire test data (or at least to a larger segment of the test data) used. It implies that the system has the characteristics of the real time markets and is robust.
  • Is it overfitting? If the system does not perform well using the test data and seems to fit only chance characteristics (not necessarily part of the test data), the system is considered to be overfitting. It is neither a robust nor reliable one and ought not to be used for trading.

If the results for the ‘out-of-sample’ years look good, continue the walk-forward process in real time to find the parameters to use with real money. Another advantage to this method of system development and trading is that your system will better adapt to changes in market behavior over time. Markets do change with time – we have all seen systems that have made money for several years and then simply stopped working because the markets have changed how often these changes affect the system is related to the best size for the training and out-of-sample set.

Find below the detailed Walk Forward Optimization results for this Moving average trading system.

Walk Forward Testing Results

Similar to Walk Forward Optimization, you should also perform Monte Carlo Analysis to check the robustness of your trading system. Check out the below article to understand it:

Monte Carlo Analysis: Step by Step Tutorial

7 Comments

  1. Thanks for this explanation, but i don’t understand what result we must keep in the historical series, the last result or that one that makes best results in robustness without distinction that it is the first or the last test ? there is no explanation about it, so walk forwar make a series of test but what of these we must use, Thanks

    • Hi Frank,

      You should use the one which shows best results in all or most of the time periods tested.

      • I’m with Frank,
        I am not sure what you a supposed to do with the data. For instance, in the results shown in your spreadsheet, the MALength1 ranges from 5 to 150, and MALength2 ranges from 10 – 195.
        What values would you use in real trading of the system described.

        Please explain in more detail, because I am not clear from your explaination

        • Hi Mark,

          You would need to manually observe walk forward results and pick the optimization variables which repeats the most. In this case it is (5,10) for MALength1 and MALength2. In case there is a tie, you would look for other backtest parameters you are interested on.

          • Dear Admin,

            Thanks for the prompt reply.
            Yes the MALength1 and MALength2 do have values of 5 and 10 most frequently in the sample data. (at the start of the sample period) However, the more recent data has much higher values for these parameters. If you take the median value for the in and out of sample data for the last 13 entries, the MALength1 and MALength2 are 32.5 and 92.5. Completely different. Do you have a more detailed explanation on interpreting walk forward outputs from Amibroker ?

          • Hi Mark,

            Yes, calculating the median is another way of interpreting walk forwards results and I have tried that too. But from my personal experience more frequent values of optimization variables has higher success probability across different markets and instruments. Having said that, there is no thumb rule to interpret walk forward results and find the best values of optimization variables. You can try different variations and arrive at the most optimum setup. Let us know too if you find something useful for our readers.

  2. Hi, i am trying one sample program in AFL, but it is always failing.
    i want to maintain values over each candle.
    Ex: HighPrice = TimeFrameGetPrice(“O”, inDaily);
    _SECTION_BEGIN(“Dynamic Support Resistance”);
    _TRACE(“HighPrice: “+HighPrice );
    MinuteAverage = (O+H+L+C)/4;
    HighPrice = IIf(MinuteAverage >= HighPrice, MinuteAverage , HighPrice);
    _SECTION_END();

    the trace is coming like this:
    HighPrice: 504 always

    it should be like this
    1. say open price is 504
    2. at 9:16, average price is 505
    then highPrice is 505
    3. at 9:17 avg is 506
    then HP: 506
    4.at 9:18 avg is 505
    then HP: 506
    like this..
    Please help me if you have any idea.

Leave a Reply

Your email address will not be published. Required fields are marked *