Collaborative Research: RI: Small: Robust Deep Learning with Big Imbalanced Data

该项目通过可扩展和强大的计算方法创新来推进人工智能（AI），从而促进科学和技术发展的进步。人工智能，尤其是深度学习，给各行各业带来了变革性的影响，并在面部识别、语音识别和机器翻译等广泛的日常技术质量上带来了巨大飞跃。然而，为了加速人工智能的民主化，仍然有许多挑战需要解决，包括数据问题和模型问题。该项目旨在通过解决与数据相关的一个关键问题来推进人工智能；即数据不平衡。当用于训练AI模型的收集数据没有足够的实例来表示模型试图学习的某些属性时，就会发生这种情况。例如，具有某种抗菌特性的分子将远远少于所有可能的分子，这使得抗菌特性的预测具有挑战性。这个项目的目标是开发有理论保证的算法，让AI从大的不平衡数据中更有效地学习。该项目还将有助于培训未来的人工智能和机器学习专业人员，包括培训高中生和代表性不足的本科生。该项目研究了深度学习中广泛的鲁棒损失。研究活动包括(i)开发可扩展的离线随机算法，用于求解不可分解的鲁棒损失，这些损失被公式化为最小-最大，最小-最小公式；（ii）开发有效的在线随机算法来解决一系列分布鲁棒优化问题，这些问题被转化为组合优化问题；（iii）通过求解被认为不可分解的鲁棒损失，开发有效的策略来训练深度神经网络；（iv）建立基础理论，包括算法的优化和统计收敛。这些算法正在对图像、图表、文本等大型不平衡数据进行评估。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project promotes the progress of science and technology development by advancing artificial intelligence (AI) through innovations in scalable and robust computational methods. AI, especially deep learning, has brought transformative impact in industries and quantum leaps in the quality of a wide range of everyday technologies including face recognition, speech recognition and machine translation. However, in order to accelerate the democratization of AI there are still many challenges to be addressed including data issues and model issues. This project seeks to advance AI by addressing one critical issue related to data; i.e., data imbalance. This happens when the collected data for training AI models does not have enough instances representing some property the models are trying to learn. For example, molecules with a certain antibacterial property would be far fewer than all possible molecules making predictions of antibacterial properties challenging. The goal of this project is to develop algorithms with theoretical guarantees to make AI learn more effectively from the big imbalanced data. This project will also contribute to training future professionals in AI and machine learning, including training high school students and under-represented undergraduates. This project investigates a broad family of robust losses for deep learning. The research activities include (i) developing scalable offline stochastic algorithms for solving non-decomposable robust losses that are formulated into min-max, min-min formulations; (ii) developing efficient online stochastic algorithms for solving a family of distributionally robust optimization problems that are cast into compositional optimization problems; (iii) developing effective strategies for training deep neural networks by solving the considered non-decomposable robust losses; (iv) establishing the underlying theory including optimization and statistical convergence of the proposed algorithms. The algorithms are being evaluated on big imbalanced data such as images, graphs, texts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.