Towards a Statistical Analysis of DNN Training Trajectories

点击翻译按钮获取中文摘要

Deep neural networks (DNNs) have become one of the state-of-the-art machine learning methods in many areas of applications. Despite this success and the fact that these methods have been considered for around 40 years, our current statistical understanding of their learning mechanisms is still rather limited. Part of the reasons for this lack of understanding is the fact that in many cases the tools of classical statistical learning theory can no longer be applied. The overall goal of this project is to establish key aspects of a statistical analysis of DNN training algorithms that are close to the ones used in practice. In particular, we will investigate the statistical properties of trajectories produced by (variants of) gradient descent, where the focus lies on the question whether such trajectories contain predictors with good generalization guarantees.