Fundamentals of Modern Machine Learning: A Precise High-dimensional Approach

随着我们渴望在日常生活的更多方面使用数据驱动的机器学习(ML)算法来创建自动决策规则，我们需要确保它们满足一些复杂的系统要求：用于自动驾驶汽车感知的ML算法需要安全，不受对手的干扰。在直接涉及人的数据的应用程序中，例如决定谁获得贷款或谁被聘用，我们需要确保公平对待我们社会中存在的人口失衡，并将其转化为数据。为了有效地在移动医疗设备等资源受限的平台上使用现代深度学习模型--这些模型越来越复杂，因此计算成本高昂--我们需要仔细平衡准确性和资源效率。我的研究计划的目标是通过开发一种现代理论来促进ML的扩展使用，该理论可以指导满足这些要求的算法的设计。发展这一理论的一个主要挑战是数据的高维性，这使得传统的统计工具变得不够用。但是，即使最近的理论捕捉到了高维性的某些方面，它们也往往未能捕捉到新发现的ML现象，因为它们产生的统计特征并不准确。为了应对这些挑战，我将为现代ML理论开发一种新的“精确高维(HD)统计”方法。我将建立一个数学框架，作为数据分布和大小、模型复杂性和算法参数的函数来精确表征分类算法的准确性。这项工作建立在我以前工作的基础上，这项工作创新了一种精确估计的方法-高清信号处理中的误差分析。现在我将应用新的框架来指导改进的ML算法的设计，考虑到三个目标：对对手扰动的稳健性(又名安全性)、对失衡的稳健性(又名公平性)和降低模型复杂性(又名资源效率)。为此，我还将开发理论驱动型统计模型，这些模型的丰富程度足以类似于数据驱动型统计模型的复杂性。本课程将为学生提供数学数据科学的基本工具：最优化、概率、统计信号处理和学习理论。正如对数据和算法中的偏见的认识是我研究的关键问题一样，我也致力于通过包容性招聘、培训环境和教学来建立一个多元化的研究小组。我的研究计划的重点与加拿大的国家人工智能战略一致，特别强调保护边缘群体权利的稳健性和公平算法。拟议计划的结果有可能导致科技行业合作，将新的经证明可靠且资源高效的算法整合到现有的数据驱动产品中。

As we aspire to use data-driven machine-learning (ML) algorithms to create automated decision rules in more aspects of everyday life, we need to make sure that they meet a number of complex system requirements: ML algorithms used for perception in self-driving cars need to be safe against disturbances caused by adversaries. In applications that directly involve data about people, such as decisions on who is granted a loan or who gets hired, we need to ensure fairness against demographic imbalances that exist in our society and translate to data. To effectively use modern deep-learning models -which are increasingly more complex, thus computationally expensive- in resource constrained platforms such as mobile health devices, we need to carefully balance accuracy and resource efficiency. The goal of my research program is to advance the expanded use of ML by developing a modern theory that can guide the design of algorithms that fulfill these requirements. A prime challenge in developing such a theory is the high-dimensionality of data that renders classical statistical tools inadequate. But even where recent theories have captured certain aspects of high-dimensionality, they have often failed to capture newly discovered ML phenomena, because they produce statistical characterizations that are not precise. To address these challenges, I will develop a new `precise high-dimensional (HD) statistics' approach to modern ML theory. I will establish a mathematical framework that will lead to precise characterization of the accuracy of classification algorithms as a function of the distribution and size of data, the model complexity, and the algorithms' parameters. This effort builds on my previous work, which innovated a method of precise estimation-error analysis in HD signal processing. Now I will apply the new framework to guide the design of improved ML algorithms with three objectives in mind: robustness to adversarial perturbations (aka safety), robustness to imbalances (aka fairness) and reduced model complexity (aka resource efficiency). To this end, I will also develop theory-driven statistical models that are rich enough to resemble the intricacies of data-driven ones. This program will provide students with the essential tools in mathematical data science: optimization, probability, statistical signal-processing, and learning theories. Just as awareness of biases in data and algorithms are key concerns of my research, I am also committed to building a diverse research group through inclusive recruitment, training environment, and teaching. The focus of my research program aligns with Canada's national strategy for AI with a particular emphasis on robustness and equitable algorithms for protecting the rights of marginalized groups. The outcomes of the proposed program have the potential to lead to tech-industry collaborations towards integrating the new provably robust and resource-efficient algorithms to existing data-driven products.