Out-of-sample Performance Estimation

量化投资的关键问题之一是投资策略的评估和选择。这通常是通过构建在给定时间点构建投资组合的算法来完成的，仅使用每个时间点的可用数据在历史期间运行该算法，这称为前进测试。然后，通过计算投资者持有生成的投资组合将获得的回报来评估该策略，最后基于该模拟估计汇总统计数据，然后在实时交易之前使用该统计数据评估该策略。一旦策略生效，就可以一天一次地收集真正的样本外性能。但是，在此过程中可能会出现各种问题。有可能以多种方式错误估计目标投资组合或将偏差引入目标投资组合，例如：多重测试(在同一数据集上测试多个策略，导致过度拟合)、认知不确定性(例如错误的模型规范)或任意不确定性(例如观察结果的可变性)。这些问题(以及其他问题)最终会导致投资者错误地估计投资策略的“样本外”(或“现场”)表现，导致他们的可用资金配置不当。本项目的目的是开发一套方法来解决这些问题，并提供更好的样本外表现估计。文献中覆盖最广泛的方法主要集中在多次测试上，但在实践中，由于要求对进行的测试进行准确的簿记，建议的方法即使不是不可能应用，也是困难的，这可能一开始就没有记录，或者对投资组合评估员来说是不可用的。多重测试问题的一个可能解决方案可能来自深度学习。如果可以综合生成其中过拟合量已知的数据集，然后在该数据集上训练网络，则新训练的网络可以应用于难以估计过拟合量的真实数据。这可以是GaN的形式，生成器创建合成数据，然后鉴别器可以对真实数据使用。Kourtis，2016)，它为夏普比率提出了统计上的激励边界和‘削发’，但只考虑静态而不是动态的情况，投资组合权重随着时间的推移而更新。为了解决这一问题，稳健优化和动态编程的工具可能会被证明是有益的。通常，稳健优化寻求在不确定情况下找到最优的“策略”，但也可以考虑给定策略在不确定情况下的表现。例如，如果收益率的分布被一种“较差”的分布所取代(通过限制两种分布之间的相对熵来控制)，那么在这种新分布下的表现会如何？最后，随机矩阵理论提供了许多有趣的工具，特别是现代的“确定性等价”的概念。这已经在随机神经网络(RNN)的样本外性能估计中得到了应用，其中找到了1层RNN的MSE的确定性等价，从而使得能够在事前估计样本外MSE。当然，投资策略的样本外绩效的估计也是这种方法的自然候选者。这个项目主要属于EPSRC‘运筹学’研究领域，但它也与‘统计学和应用概率’研究领域等有关。该项目得到量化对冲基金Qube Research&Technologies(QRT)的支持，QRT研究主管Marco Dion负责监督，并受益于与QRT其他研究人员的讨论。

One of the key issues in quantitative investing is the assessment and selection of investment strategies. This is typically done by building an algorithm to construct a portfolio at a given point in time, running this algorithm over a historical period using only the available data at each point in time, this is known as walk-forward testing. The strategy is then assessed by computing what the return would have been given the investor had held the generated portfolio, and finally estimating summary statistics based on this simulation, which will then be used to assess the strategy prior to live trading. Once the strategy is live, true out-of-sample performance can be collected one day at a time.However, various issues can arise in this procedure. It is possible to misestimate or introduce bias into the target portfolio in a number of ways, such as: multiple testing (by which multiple strategies are tested on the same dataset, leading to overfitting), epistemic uncertainty (e.g. incorrect model specification) or aleatoric uncertainty (e.g. variability in the observed outcome). These issues (among others) culminate in causing an investor to incorrectly estimate what the 'out-of-sample' (or 'live') performance of an investment strategy will be, leading to misallocation of their available capital.The aim of this project is to develop a set of methods to account for these issues and provide better estimates of out-of-sample performance. The most widely covered methods in the literature primarily focus on multiple testing, but in practice the proposed methods are difficult if not impossible to apply due to requiring accurate bookkeeping of the tests conducted, which may either be not recorded in the first place, or not available to the portfolio assessor. One possible solution to the multiple testing problem could come from deep learning. If it is possible to synthetically generate a dataset where the amount of overfitting is known and then train a network on this dataset, then the newly trained net can be applied to real data, where the amount of overfitting is difficult to estimate. This could be in the form of a GAN, whereby the generator creates the synthetic data and the discriminator can then be used on the real data.There is a small body of literature, (Kan & Zhou, 2007. Kourtis, 2016) which suggests statistically motivated bounds and 'haircuts' for the Sharpe Ratio, but solely consider the static rather than dynamic case, where portfolio weights are updated through time. To solve this, tools from robust optimisation and dynamic programming may prove beneficial. Typically, robust optimisation seeks to find an optimal 'strategy' under uncertainty, however it is also possible to consider what the performance of a given strategy would be under uncertainty. For example, if the distribution of the returns is replaced by a 'worse' distribution (controlled by limiting the relative entropy between the two distributions), what will the performance be under this new distribution?Finally, Random Matrix Theory provides a number of interesting tools, in particular the modern concept of `deterministic equivalents'. This has already seen applications in the estimation of out-of-sample performance for Random Neural Networks (RNN), where a deterministic equivalent for the MSE of a 1-layer RNN was found thus enabling the estimation of the out-of-sample MSE ex-ante. Naturally, the estimation of out-of-sample performance for investment strategies is a natural candidate for this method also.This project primarily falls within the EPSRC 'Operational research' research area, however it also has connections to the 'Statistics and applied probability' research area, among others. The project is supported by Qube Research & Technologies (QRT) a quantitative hedge fund, with supervision from Marco Dion, Head of Research, at QRT and benefits from discussion with other researchers at QRT.