How to Improve the Probability of Backtesting Strategy

Want to increase your chances of success in the markets? Backtesting can help by testing your trading strategies on historical data. Follow our tips on how to effectively conduct backtesting, including using multiple data sets, considering relevant time periods and samples, and avoiding bias and data dredging. This valuable tool is useful for both systematic and discretionary traders and can help you make informed decisions about your strategies.

There are far too many factors involved when it comes to making money in the markets. Several strategies are thought of, but few will actually give you the ‘Eureka!’ moment. Therefore, traders should practice backtesting before the implementation of any strategy using historical data – be it automatic or manual trading – to boost the probability of success. Since we cannot see or predict the future, especially when so many variables are involved, the next best thing we can do is to know how a model behaves when certain past data sets are input into it. Backtesting, thus, is fast emerging as an essential tool. 

What is Backtesting?

Backtesting Feature on uTrade algos

Backtesting is the process which sees how well a strategy or model would have done ex-post. With the help of this, by using a set of historical data, one can check the viability of a trading strategy. If a trading strategy works during backtesting, one can also be confident of implementing it going forward. Added to this, if too many conditions are not fitted into the model, it can also prove to be effective in future scenarios. Hence, both systematic and discretionary traders have begun to consider it as a very useful tool. 

Why Backtesting?

Backtesting helps one to find how a particular strategy will pan out in the future. This is done using historical data to check how it behaves with these data sets. The theory behind this is that if it has worked well/poorly in the past, it will do so in the future as well. That is, if​​ a well-conducted backtest gives a positive result, it means that the model at hand is sound enough to be used and is likely to yield profits when implemented. On the other hand, if the strategy gives suboptimal results, it should be rejected and not used in reality, as it will most likely result in losses.  

Things to Note While Checking if the Trading Strategy Works

Using More Than One Data Set

Backtesting uses historical data. Hence, using only a single data set is not advisable. To see the viability of a model, one should test it on different time periods or use out-of-sample data to confirm its potential viability.

Using the Correct Sample

Relevant time period :

When backtesting, the sample data must be chosen from a relevant time period which reflects the market conditions accurately. Only then can one judge if the backtest results represent a fluke or a sound trading decision.

Relevant data set :

The historical data set that is being used should have a true representative sample of stocks, which includes those of companies that eventually went bankrupt, were sold, or were liquidated. If data from only those companies are used which are still in existence in the present time, then the results that the test will yield will be artificially high returns.

All Trading Costs Should be Considered

No matter how insignificant, when backtesting, all trading costs should be taken into consideration. Else the results can show a profit/loss, which is not a true representation. Hence, when the model is being prepared, a trader must ensure that his backtesting software takes into account all costs.

Conduct Scenario Analysis 

Often, so as to ensure that the backtested results are the correct strategy, traders also use scenario analysis. This is a process wherein hypothetical data is used to see what the outcomes are. This helps a trader to find out how the values of a portfolio will change in case of the occurrence of an unfavourable event. The results of this, together with the backtested one, are then analysed together to come to a decision on whether a strategy is viable or not. 

Avoid Bias

If backtesting has to provide meaningful results, the testing has to be done where no bias creeps in. This means the development of the strategy should happen without relying on the data used in backtesting – the model should be built using data that is different from the ones that are being used for testing. Else, the results of the test will mean nothing. 

Avoid Data Dredging

In data dredging, traders use the same data set to test several hypothetical strategies. This might result in giving outcomes that show success during the testing, whereas they are likely to fail in real-time markets since many invalid strategies might land up beating the market over a given period of time. One way to avoid this is by picking a strategy that works for a given time period when an in-sample is used and then using that in a different time period or an out-of-sample. If the results then, too, stay the same, then the strategy can be considered to be a viable one. 

Avoid Over-optimisation

Optimising a model makes it more efficient. After an algorithm is designed, beta testing happens, wherein parameters are added and/or removed so that the model works best. However, often, over-optimisation occurs, which makes the strategy not reflect the picture accurately when data is input into it. Excessive adjustment of the variables of an algorithm is what leads to over-optimisation. Often this is done so that the program’s performance is maximised. In the process, though, it no longer reflects the market picture accurately.

‍Summarizing why Backtesting your trading strategies is necessay

There are always factors like luck, randomness, trading psychology, and unpredictable transaction costs, which can make the real-time results far different from what backtesting revealed. Having said this, backtesting has come to play a significant part in a trader’s strategy development. If performed while keeping in mind the aforementioned tips, it will help in creating sound strategies that can be used in live markets.