Towards a Statistical Analysis of DNN Training Trajectories

深度神经网络（dnn）已成为许多应用领域中最先进的机器学习方法之一。尽管取得了这样的成功，而且这些方法已经被考虑了大约40年，但我们目前对它们的学习机制的统计理解仍然相当有限。这种缺乏理解的部分原因是，在许多情况下，经典统计学习理论的工具不再适用。该项目的总体目标是建立接近实践中使用的DNN训练算法的统计分析的关键方面。特别是，我们将研究由梯度下降（变体）产生的轨迹的统计特性，其中重点在于这些轨迹是否包含具有良好泛化保证的预测因子。

Deep neural networks (DNNs) have become one of the state-of-the-art machine learning methods in many areas of applications. Despite this success and the fact that these methods have been considered for around 40 years, our current statistical understanding of their learning mechanisms is still rather limited. Part of the reasons for this lack of understanding is the fact that in many cases the tools of classical statistical learning theory can no longer be applied. The overall goal of this project is to establish key aspects of a statistical analysis of DNN training algorithms that are close to the ones used in practice. In particular, we will investigate the statistical properties of trajectories produced by (variants of) gradient descent, where the focus lies on the question whether such trajectories contain predictors with good generalization guarantees.