Out-of-sample Performance Estimation

量化投资的关键问题之一是投资策略的评估和选择。这通常是通过构建一个算法来在给定的时间点构建一个投资组合，在一个历史时期内只使用每个时间点的可用数据运行该算法来完成的，这被称为向前走测试。然后通过计算投资者持有生成的投资组合的回报来评估策略，并最终基于此模拟估计汇总统计数据，然后将用于在实时交易之前评估策略。一旦策略生效，就可以每天收集真实的样本外绩效。然而，在此过程中可能会出现各种问题。有可能以多种方式错误估计或将偏差引入目标投资组合，例如：多重测试（在同一数据集上测试多个策略，导致过度拟合），认识不确定性（例如不正确的模型规格）或任意不确定性（例如观察结果的可变性）。这些问题（除其他外）最终导致投资者错误地估计投资策略的“样本外”（或“活”）表现，导致其可用资本的错误分配。本项目的目的是开发一套方法来解决这些问题，并提供更好的样本外表现估计。在文献中最广泛的方法主要集中在多重测试，但在实践中，所提出的方法是困难的，如果不是不可能应用，由于需要准确的簿记进行的测试，这可能是没有记录在首位，或无法提供给投资组合评估。多重测试问题的一个可能解决方案可能来自深度学习。如果可以合成地生成过拟合量已知的数据集，然后在该数据集上训练网络，则新训练的网络可以应用于真实的数据，其中过拟合量难以估计。这可以是GAN的形式，其中生成器创建合成数据，然后可以将合成数据用于真实的数据。Kourtis，2016年），它建议夏普比率的统计动机边界和“理发”，但只考虑静态而不是动态的情况，其中投资组合权重随时间更新。为了解决这个问题，来自鲁棒优化和动态规划的工具可能是有益的。通常，鲁棒优化寻求在不确定性下找到最优“策略”，但是也可以考虑给定策略在不确定性下的性能。例如，如果收益的分布被一个“更差”的分布（通过限制两个分布之间的相对熵来控制）所取代，那么在这个新分布下的表现会是什么？最后，随机矩阵理论提供了一些有趣的工具，特别是“确定性等价物”的现代概念。这已经在随机神经网络（RNN）的样本外性能估计中得到了应用，其中发现了1层RNN的MSE的确定性等效，从而能够预先估计样本外MSE。当然，投资策略的样本外业绩估计也是这种方法的自然候选者。这个项目主要福尔斯属于EPSRC的“运筹学”研究领域，但它也与“统计和应用概率”研究领域有联系。该项目得到了Qube Research & Technologies（QRT）的支持，QRT研究主管Marco Dion的监督，并受益于与QRT其他研究人员的讨论。

One of the key issues in quantitative investing is the assessment and selection of investment strategies. This is typically done by building an algorithm to construct a portfolio at a given point in time, running this algorithm over a historical period using only the available data at each point in time, this is known as walk-forward testing. The strategy is then assessed by computing what the return would have been given the investor had held the generated portfolio, and finally estimating summary statistics based on this simulation, which will then be used to assess the strategy prior to live trading. Once the strategy is live, true out-of-sample performance can be collected one day at a time.However, various issues can arise in this procedure. It is possible to misestimate or introduce bias into the target portfolio in a number of ways, such as: multiple testing (by which multiple strategies are tested on the same dataset, leading to overfitting), epistemic uncertainty (e.g. incorrect model specification) or aleatoric uncertainty (e.g. variability in the observed outcome). These issues (among others) culminate in causing an investor to incorrectly estimate what the 'out-of-sample' (or 'live') performance of an investment strategy will be, leading to misallocation of their available capital.The aim of this project is to develop a set of methods to account for these issues and provide better estimates of out-of-sample performance. The most widely covered methods in the literature primarily focus on multiple testing, but in practice the proposed methods are difficult if not impossible to apply due to requiring accurate bookkeeping of the tests conducted, which may either be not recorded in the first place, or not available to the portfolio assessor. One possible solution to the multiple testing problem could come from deep learning. If it is possible to synthetically generate a dataset where the amount of overfitting is known and then train a network on this dataset, then the newly trained net can be applied to real data, where the amount of overfitting is difficult to estimate. This could be in the form of a GAN, whereby the generator creates the synthetic data and the discriminator can then be used on the real data.There is a small body of literature, (Kan & Zhou, 2007. Kourtis, 2016) which suggests statistically motivated bounds and 'haircuts' for the Sharpe Ratio, but solely consider the static rather than dynamic case, where portfolio weights are updated through time. To solve this, tools from robust optimisation and dynamic programming may prove beneficial. Typically, robust optimisation seeks to find an optimal 'strategy' under uncertainty, however it is also possible to consider what the performance of a given strategy would be under uncertainty. For example, if the distribution of the returns is replaced by a 'worse' distribution (controlled by limiting the relative entropy between the two distributions), what will the performance be under this new distribution?Finally, Random Matrix Theory provides a number of interesting tools, in particular the modern concept of `deterministic equivalents'. This has already seen applications in the estimation of out-of-sample performance for Random Neural Networks (RNN), where a deterministic equivalent for the MSE of a 1-layer RNN was found thus enabling the estimation of the out-of-sample MSE ex-ante. Naturally, the estimation of out-of-sample performance for investment strategies is a natural candidate for this method also.This project primarily falls within the EPSRC 'Operational research' research area, however it also has connections to the 'Statistics and applied probability' research area, among others. The project is supported by Qube Research & Technologies (QRT) a quantitative hedge fund, with supervision from Marco Dion, Head of Research, at QRT and benefits from discussion with other researchers at QRT.