Multi-parameter regularization in high-dimensional learning

在许多系统中，准确的预测是实现成本节约、效率、健康、安全和组织目的的关键因素。在对稳健预测方法日益增长的需求的鼓舞下，在这个联合的国际项目中，我们正在开发一种技术和数值方法的全面分析，以便从粗略测量的高维数据执行可靠的预测。应克服处理高维噪声数据的挑战，方法是在现有数据的基础上加入额外的信息，通过多参数正则化进行优化，并研究不同的候选核心模型以及额外的约束集。我们具体阐述了三个基本目标，前两个目标具有方法论性质，后一个目标具有应用性。第一个目标是在Banach空间中发展多罚正则化的综合理论和数值方法，这些空间可以是再生核Banach空间或稀疏表示函数的空间。这是由于高维数据的几何/结构特征在很大程度上是预期的，这些特征在希尔伯特空间(通常是更各向同性的)框架中可能不能很好地表示。第二个目标是将Banach空间中的多惩罚正则化应用于高维监督学习。在这里，我们通过假设我们的函数有一个特殊的表示/格式来关注两个主要的降维机制，然后我们将学习问题重塑到具有自适应选择的参数的多惩罚正则化框架中。作为最后一个目标，我们将把前两个任务中开发的方法应用于元学习，以实现算法的最佳参数选择。由于在许多算法中，出于数值模拟的目的，但更关键的是在数据分析中，需要调整某些参数以优化性能，以速度或所产生的(近似)质量来测量，这要求开发参数的快速选择规则，可能提供数据的某些特征，其仍然可以保持相当高的维度。该规则应通过对算法以前的应用进行培训来学习。在高维学习的背景下，这一问题似乎还没有得到系统的研究。上述项目指导可能在未来成为正则化、学习和逼近理论之间的坚实桥梁，并可以为各种实际应用发挥基础作用。

Making accurate predictions is a crucial factor in many systems for cost savings, efficiency, health, safety, and organizational purposes. Inspired by the increased demand of robust predictive methods, in this joint international project we are developing a comprehensive analysis of techniques and numerical methods for performing reliable predictions from roughly measured high-dimensional data. The challenge of working with high-dimensional noisy data shall be overcome by incorporating additional information on top of the available data, through optimization by means of multi-parameter regularization, and studying different candidate core models together with additional sets of constraints. We address specifically three fundamental objectives, the first two of them have methodological nature and the last one has applicative nature. The first objective is to develop both comprehensive theoretical and numerical approaches to multi-penalty regularization in Banach spaces, which may be reproducing kernel Banach spaces or spaces of sparsely represented functions. This is motivated by the largely expected geometrical/structured features of high-dimensional data, which may not be well-represented in the framework of (typically more isotropic) Hilbert spaces. Moreover, it is a rather open research field where only preliminary results are available.The second objective will be to use multi-penalty regularization in Banach spaces in high-dimensional supervised learning. Here we focus on two main mechanisms of dimensionality reduction by assuming that our function has a special representation / format and then we recast the learning problem into the framework of multi-penalty regularization with the adaptively chosen parameters. As the last objective we shall apply the methodologies developed in the previous two tasks to meta-learning for optimal parameter choices of algorithms. Since in many algorithms, for numerical simulation purposes, but even more crucially in data analysis, certain parameters need to be tuned for optimal performances, measured in terms either of speed or of resulting (approximation) quality, this begs for the development of a fast choice rule for the parameters, possibly provided certain features of the data, which may retain nevertheless a rather high dimensionality. This rule shall be learned by training on previous applications of the algorithm. It appears that this issue has not been systematically studied in the context of high-dimensional learning.The above mentioned project directions may, in the future, serve as a solid bridge across regularization, learning, and approximation theories and can play a fundamental role for various practical applications.