Collaborative Research: RI: Small: Robust Deep Learning with Big Imbalanced Data

合作研究：RI：小型：具有大不平衡数据的鲁棒深度学习

• 批准号: 2110545
• 负责人: Tianbao Yang
• 金额: $ 26.43万
• 依托单位: University of Iowa
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2022-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2110545&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Robust

This project promotes the progress of science and technology development by advancing artificial intelligence (AI) through innovations in scalable and robust computational methods. AI, especially deep learning, has brought transformative impact in industries and quantum leaps in the quality of a wide range of everyday technologies including face recognition, speech recognition and machine translation. However, in order to accelerate the democratization of AI there are still many challenges to be addressed including data issues and model issues. This project seeks to advance AI by addressing one critical issue related to data; i.e., data imbalance. This happens when the collected data for training AI models does not have enough instances representing some property the models are trying to learn. For example, molecules with a certain antibacterial property would be far fewer than all possible molecules making predictions of antibacterial properties challenging. The goal of this project is to develop algorithms with theoretical guarantees to make AI learn more effectively from the big imbalanced data. This project will also contribute to training future professionals in AI and machine learning, including training high school students and under-represented undergraduates. This project investigates a broad family of robust losses for deep learning. The research activities include (i) developing scalable offline stochastic algorithms for solving non-decomposable robust losses that are formulated into min-max, min-min formulations; (ii) developing efficient online stochastic algorithms for solving a family of distributionally robust optimization problems that are cast into compositional optimization problems; (iii) developing effective strategies for training deep neural networks by solving the considered non-decomposable robust losses; (iv) establishing the underlying theory including optimization and statistical convergence of the proposed algorithms. The algorithms are being evaluated on big imbalanced data such as images, graphs, texts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目通过可扩展和强大的计算方法的创新来推进人工智能（AI），从而促进科学技术的进步。人工智能，尤其是深度学习，给各行各业带来了变革性影响，并为人脸识别、语音识别和机器翻译等各种日常技术的质量带来了质的飞跃。然而，为了加速人工智能的民主化，仍然有许多挑战需要解决，包括数据问题和模型问题。该项目旨在通过解决与数据相关的一个关键问题来推进人工智能的发展；即数据不平衡。当收集的用于训练 AI 模型的数据没有足够的实例来表示模型试图学习的某些属性时，就会发生这种情况。例如，具有某种抗菌特性的分子将远远少于所有可能的分子，这使得抗菌特性的预测具有挑战性。该项目的目标是开发有理论保证的算法，使人工智能更有效地从不平衡的大数据中学习。该项目还将有助于培训未来的人工智能和机器学习专业人士，包括培训高中生和代表性不足的本科生。 该项目研究了深度学习的一系列稳健损失。研究活动包括（i）开发可扩展的离线随机算法，用于解决不可分解的鲁棒损失，这些损失被制定为最小-最大、最小-最小公式； (ii) 开发高效的在线随机算法来解决一系列分布稳健的优化问题，这些问题被转化为组合优化问题； (iii) 通过解决所考虑的不可分解鲁棒损失来制定训练深度神经网络的有效策略； (iv) 建立基础理论，包括所提出算法的优化和统计收敛。这些算法正在根据图像、图表、文本等大的不平衡数据进行评估。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Online Method for A Class of Distributionally Robust Optimization with Non-convex Objectives

• 发表时间: 2020-06
• 作者: Qi Qi-Qi;Zhishuai Guo;Yi Xu;Rong Jin;Tianbao Yang

An Online Method for Distributionally Deep Robust Optimization

• 发表时间: 2020-06
• 期刊: arXiv: Learning
• 作者: Qi Qi-Qi;Zhishuai Guo;Yi Xu;Rong Jin;Tianbao Yang