CIF: Small: Alpha Loss: A New Framework for Understanding and Trading Off Computation, Accuracy, and Robustness in Machine Learning

点击翻译按钮获取中文摘要

At the heart of the machine learning (ML) and artificial intelligence (AI) revolution are models that are trained using vast amounts of data. Given the increasing use of such data-driven modeling, there is an urgent need to understand and leverage the tradeoffs between various performance characteristics such as accuracy (statistical efficiency), computational speed (computational efficiency), and robustness (say to noise, adversarial tampering, and imbalance or biased data). This project develops a unified and powerful framework for understanding and trading off these facets by introducing the family of alpha-loss functions -- often-used loss functions such as the 0-1 loss, the log-loss, and the exponential-loss appear as instantiations of the alpha-loss framework. Over the past few years, we have seen a steadily growing recognition amongst advocates, regulators, and scientists that data-driven inference and decision engines pose significant challenges for ensuring non-discrimination, and fair and inclusive representation. The alpha-loss framework, combined with several technological advances, will allow practitioners to incorporate fairness as an explicit knob to be tuned during the development of machine learning models. Broader impacts of this work also include developing ML modules for a week-long summer camp for high school students as well as providing research opportunities for such students. This project: (i) develops theoretical results on the behavior of the loss landscape as a function of the tuning parameter alpha, thereby illuminating the value and limitation of the industry standard log-loss, (ii) establishes accuracy-speed tradeoffs and generalization bounds, and (iii) designs practical adaptive algorithms with guarantees for tuning the hyperparameter alpha to achieve various operating points along the tradeoff. This project establishes the robustness properties of alpha-loss via the theory of influence functions. By introducing much-needed models for noise and adversarial examples, this work develops a principled method to choose alpha slightly larger than 1 to design models more robust to noise and adversaries. Using both influence functions and constrained learning settings such as fair classification, this project studies the efficacy of tuning alpha below one in order to enhance sensitivity to limited samples in highly imbalanced training datasets. Finally, this project also develops alpha-Boost as a tunable boosting algorithm with guaranteed convergence, robustness to noise and, where needed, online adaptation. Research is enhanced at every stage of this project through rigorous testing of algorithms on both synthetic and publicly available real datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.