Out-of-sample Performance Estimation

点击翻译按钮获取中文摘要

One of the key issues in quantitative investing is the assessment and selection of investment strategies. This is typically done by building an algorithm to construct a portfolio at a given point in time, running this algorithm over a historical period using only the available data at each point in time, this is known as walk-forward testing. The strategy is then assessed by computing what the return would have been given the investor had held the generated portfolio, and finally estimating summary statistics based on this simulation, which will then be used to assess the strategy prior to live trading. Once the strategy is live, true out-of-sample performance can be collected one day at a time.However, various issues can arise in this procedure. It is possible to misestimate or introduce bias into the target portfolio in a number of ways, such as: multiple testing (by which multiple strategies are tested on the same dataset, leading to overfitting), epistemic uncertainty (e.g. incorrect model specification) or aleatoric uncertainty (e.g. variability in the observed outcome). These issues (among others) culminate in causing an investor to incorrectly estimate what the 'out-of-sample' (or 'live') performance of an investment strategy will be, leading to misallocation of their available capital.The aim of this project is to develop a set of methods to account for these issues and provide better estimates of out-of-sample performance. The most widely covered methods in the literature primarily focus on multiple testing, but in practice the proposed methods are difficult if not impossible to apply due to requiring accurate bookkeeping of the tests conducted, which may either be not recorded in the first place, or not available to the portfolio assessor. One possible solution to the multiple testing problem could come from deep learning. If it is possible to synthetically generate a dataset where the amount of overfitting is known and then train a network on this dataset, then the newly trained net can be applied to real data, where the amount of overfitting is difficult to estimate. This could be in the form of a GAN, whereby the generator creates the synthetic data and the discriminator can then be used on the real data.There is a small body of literature, (Kan & Zhou, 2007. Kourtis, 2016) which suggests statistically motivated bounds and 'haircuts' for the Sharpe Ratio, but solely consider the static rather than dynamic case, where portfolio weights are updated through time. To solve this, tools from robust optimisation and dynamic programming may prove beneficial. Typically, robust optimisation seeks to find an optimal 'strategy' under uncertainty, however it is also possible to consider what the performance of a given strategy would be under uncertainty. For example, if the distribution of the returns is replaced by a 'worse' distribution (controlled by limiting the relative entropy between the two distributions), what will the performance be under this new distribution?Finally, Random Matrix Theory provides a number of interesting tools, in particular the modern concept of `deterministic equivalents'. This has already seen applications in the estimation of out-of-sample performance for Random Neural Networks (RNN), where a deterministic equivalent for the MSE of a 1-layer RNN was found thus enabling the estimation of the out-of-sample MSE ex-ante. Naturally, the estimation of out-of-sample performance for investment strategies is a natural candidate for this method also.This project primarily falls within the EPSRC 'Operational research' research area, however it also has connections to the 'Statistics and applied probability' research area, among others. The project is supported by Qube Research & Technologies (QRT) a quantitative hedge fund, with supervision from Marco Dion, Head of Research, at QRT and benefits from discussion with other researchers at QRT.