Fundamentals of Modern Machine Learning: A Precise High-dimensional Approach

点击翻译按钮获取中文摘要

As we aspire to use data-driven machine-learning (ML) algorithms to create automated decision rules in more aspects of everyday life, we need to make sure that they meet a number of complex system requirements: ML algorithms used for perception in self-driving cars need to be safe against disturbances caused by adversaries. In applications that directly involve data about people, such as decisions on who is granted a loan or who gets hired, we need to ensure fairness against demographic imbalances that exist in our society and translate to data. To effectively use modern deep-learning models -which are increasingly more complex, thus computationally expensive- in resource constrained platforms such as mobile health devices, we need to carefully balance accuracy and resource efficiency. The goal of my research program is to advance the expanded use of ML by developing a modern theory that can guide the design of algorithms that fulfill these requirements. A prime challenge in developing such a theory is the high-dimensionality of data that renders classical statistical tools inadequate. But even where recent theories have captured certain aspects of high-dimensionality, they have often failed to capture newly discovered ML phenomena, because they produce statistical characterizations that are not precise. To address these challenges, I will develop a new `precise high-dimensional (HD) statistics' approach to modern ML theory. I will establish a mathematical framework that will lead to precise characterization of the accuracy of classification algorithms as a function of the distribution and size of data, the model complexity, and the algorithms' parameters. This effort builds on my previous work, which innovated a method of precise estimation-error analysis in HD signal processing. Now I will apply the new framework to guide the design of improved ML algorithms with three objectives in mind: robustness to adversarial perturbations (aka safety), robustness to imbalances (aka fairness) and reduced model complexity (aka resource efficiency). To this end, I will also develop theory-driven statistical models that are rich enough to resemble the intricacies of data-driven ones. This program will provide students with the essential tools in mathematical data science: optimization, probability, statistical signal-processing, and learning theories. Just as awareness of biases in data and algorithms are key concerns of my research, I am also committed to building a diverse research group through inclusive recruitment, training environment, and teaching. The focus of my research program aligns with Canada's national strategy for AI with a particular emphasis on robustness and equitable algorithms for protecting the rights of marginalized groups. The outcomes of the proposed program have the potential to lead to tech-industry collaborations towards integrating the new provably robust and resource-efficient algorithms to existing data-driven products.