Non-parametric estimation under covariate shift: From fundamental bounds to efficient algorithms

在过去的十年中，用于大规模预测的机器学习（ML）方法对科学和工程的各个分支产生了巨大的影响。 然而，当这些方法用于与训练数据不同的数据集时，通常被报告为不稳定，或者以神秘的方式失败。 这种类型的失败--以及我们对其根本原因的缺乏了解--对在高风险环境中采用现代ML方法构成了主要障碍，其中失败的成本可能很高（例如，自动驾驶汽车、金融风险评估、医疗诊断）。 该研究项目的目标是描述此类故障的根本原因，并开发新的算法来缓解这些问题。该项目还将通过以下方式将研究与教育结合起来：（a）让本科生和研究生参与研究和传播研究成果;（B）将研究成果纳入麻省理工学院的课程和网上课程材料，其他大学也可以使用这些材料;（c）在暑期学校和讲习班开设短期课程。该项目还将支持指导STEM领域代表性不足的研究生和博士后，并在其职业生涯中提供持续的专业支持。更详细地说，研究重点是协变量转移的挑战，其中用于训练大规模预测模型的特征向量的分布与评估模型的特征向量的分布不同。 一个初始目标是在有限样本非渐近环境下，描述基本极限的特征，并在协变量移位下开发有效的估计算法。 有了这样的理解，后续的目标是开发计算效率高的程序，实现基本的限制沿着他们的行为的理论理解。 该项目的工作将利用和建立在非参数分析、经验过程理论、测量集中、再生核希尔伯特空间和信息理论的技术和工具之上。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning (ML) methods for large-scale prediction have had dramatic impacts on various branches of science and engineering over the past decade. However, these methods --- when they are used on data sets that differ from the training data --- are often reported to be unstable, or to fail in mysterious ways. Failures of this type --- and our lack of understanding of their root causes --- pose a major roadblock to the adoption of modern ML methods in high-stake settings, where the cost of failure might be significant (e.g., self-driving cars, financial risk assessments, medical diagnoses). The goal of this research project is to characterize the fundamental causes of such failures, and to develop new algorithms that mitigate these issues. The project will also integrate research and education through: (a) the involvement of both undergraduate and graduate students in the research and in the dissemination of research results; (b) the inclusion of the research results in courses at MIT and in the web-based course materials, which are accessed from other universities; and (c) short courses at summer schools and workshops. The project will also support mentoring graduate students and postdocs that are under-represented in the STEM fields, with continued professional support in their careers.In more detail, the research focuses on the challenge of covariate shift, in which the distribution of the feature vectors used to train a large-scale prediction model differ from those on which it is evaluated. An initial goal is to characterize fundamental limits and develop efficient algorithms for estimation under covariate shift, in the finite-sample non-asymptotic setting. Armed with such an understanding, a follow-up goal is to develop computationally efficient procedures that achieve the fundamental limits along with theoretical understanding of their behavior. The work in this project will leverage and build upon techniques and tools from non-parametric analysis, empirical process theory, concentration of measure, reproducing kernel Hilbert spaces, and information theory.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.