Multi-parameter regularization in high-dimensional learning

在许多系统中，为了节省成本、效率、健康、安全和组织目的，做出准确的预测是一个至关重要的因素。受到对可靠预测方法日益增长的需求的启发，在这个联合国际项目中，我们正在开发一种综合分析技术和数值方法，以便从粗略测量的高维数据中进行可靠的预测。要克服处理高维噪声数据的挑战，需要在现有数据的基础上加入额外的信息，通过多参数正则化方法进行优化，并结合额外的约束集研究不同的候选核心模型。我们具体解决三个基本目标，其中前两个具有方法论性质，最后一个具有应用性质。第一个目标是发展巴拿赫空间中多惩罚正则化的综合理论和数值方法，这可能是再现核巴拿赫空间或稀疏表示函数的空间。这是由高维数据的几何/结构特征所驱动的，这些特征在希尔伯特空间的框架（通常更各向同性）中可能无法很好地表示。此外，这是一个相当开放的研究领域，只有初步的结果。第二个目标是在高维监督学习的Banach空间中使用多惩罚正则化。本文通过假设我们的函数具有特殊的表示/格式，重点讨论了两种主要的降维机制，然后我们将学习问题重新定义为具有自适应选择参数的多惩罚正则化框架。作为最后一个目标，我们将把前两个任务中开发的方法应用到元学习中，以实现算法的最优参数选择。因为在许多算法中，为了数值模拟的目的，但更重要的是在数据分析中，需要调整某些参数以获得最佳性能，以速度或结果（近似）质量来衡量，这就要求为参数开发一个快速选择规则，可能提供数据的某些特征，但可能保留相当高的维度。这条规则需要通过对算法之前应用的训练来学习。在高维学习的背景下，这个问题似乎还没有得到系统的研究。在未来，上述项目方向可以作为跨越正则化、学习和近似理论的坚实桥梁，并可以在各种实际应用中发挥基础作用。

Making accurate predictions is a crucial factor in many systems for cost savings, efficiency, health, safety, and organizational purposes. Inspired by the increased demand of robust predictive methods, in this joint international project we are developing a comprehensive analysis of techniques and numerical methods for performing reliable predictions from roughly measured high-dimensional data. The challenge of working with high-dimensional noisy data shall be overcome by incorporating additional information on top of the available data, through optimization by means of multi-parameter regularization, and studying different candidate core models together with additional sets of constraints. We address specifically three fundamental objectives, the first two of them have methodological nature and the last one has applicative nature. The first objective is to develop both comprehensive theoretical and numerical approaches to multi-penalty regularization in Banach spaces, which may be reproducing kernel Banach spaces or spaces of sparsely represented functions. This is motivated by the largely expected geometrical/structured features of high-dimensional data, which may not be well-represented in the framework of (typically more isotropic) Hilbert spaces. Moreover, it is a rather open research field where only preliminary results are available.The second objective will be to use multi-penalty regularization in Banach spaces in high-dimensional supervised learning. Here we focus on two main mechanisms of dimensionality reduction by assuming that our function has a special representation / format and then we recast the learning problem into the framework of multi-penalty regularization with the adaptively chosen parameters. As the last objective we shall apply the methodologies developed in the previous two tasks to meta-learning for optimal parameter choices of algorithms. Since in many algorithms, for numerical simulation purposes, but even more crucially in data analysis, certain parameters need to be tuned for optimal performances, measured in terms either of speed or of resulting (approximation) quality, this begs for the development of a fast choice rule for the parameters, possibly provided certain features of the data, which may retain nevertheless a rather high dimensionality. This rule shall be learned by training on previous applications of the algorithm. It appears that this issue has not been systematically studied in the context of high-dimensional learning.The above mentioned project directions may, in the future, serve as a solid bridge across regularization, learning, and approximation theories and can play a fundamental role for various practical applications.