CAREER: Continual Learning with Evolving Memory, Soft Supervision, and Cross-Domain Knowledge - Foundational Theory and Advanced Algorithms

随着数据和学习环境的迅速扩展，人工智能的应用取决于其扩展和适应的能力。作为回应，持续学习正在成为满足这一需求的一个有希望的范例。与其他学习范式不同，持续学习强调在无缝整合新信息的同时保持先前学习任务的性能的能力。它注重通过适当地回忆过去的知识和积极地寻找侧面信息来加速学习和提高准确性，从而快速适应新环境。然而，目前的持续学习方法主要是经验性的，缺乏明确的理论基础，限制了其更广泛的应用，阻碍了进一步的发展。该项目旨在通过开发一个持续学习的原则框架来弥合这一差距，该框架由新颖的理论见解和实用的算法设计组成。研究成果将极大地促进我们对持续学习的理解，丰富其算法框架，并提供可扩展的算法包。研究成果将被整合到各级机器学习课程中，使电气工程、统计学和计算机科学等学科的学生受益。该项目将积极让代表性不足的学生参与STEM，在本科和研究生阶段协同研究和教育，并通过人工智能学徒计划为K-12学生开发入门材料。该项目的总体目标是为自主机器学习者导航动态数据环境的持续学习开创一个方法论框架。该框架涉及持续学习的三个基本方面。首先，它将建立持续学习和可扩展在线算法的数学基础，以重新收集快速适应和能力扩展的不断发展的记忆。其次，它将开发方法，从信息丰富、甚至不太可靠的伪标签中提高学习效率。最后，它将开发通过拥有跨模态数据的同伴学习者的任务特定帮助来扩展预测能力的方法。发展的基础将导致新的算法，使智能工程系统在不断变化的世界中更有效地学习。该项目还将为关键的机器学习问题提供深入的见解，例如理解动态学习环境中适应性和遗忘之间的基本权衡，衡量轻微但共同依赖的数据扰动对建模结果的最大影响，以及在潜在的概率规律仍然难以捉摸时量化模型的改进空间。此外，预计这些发展将产生更广泛的影响，使资源受限的机器学习者能够在复杂的实际应用中不断扩大他们的学习能力，例如智能交通系统和蜂窝性能管理。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As the landscape of data and learning environments expands rapidly, the utility of artificial intelligence hinges on its ability to scale and adapt. In response, continual learning is emerging as a promising paradigm to meet this demand. Distinct from other learning paradigms, continual learning emphasizes the ability to maintain performance on previously learned tasks while seamlessly integrating new information. It focuses on rapid adaptation to new environments by appropriately recalling past knowledge and actively seeking side information to accelerate learning and improve accuracy. However, current methods in continual learning are predominantly empirical and lack a clear theoretical foundation, limiting their wider application and hindering further progress. This project aims to bridge this gap by developing a principled framework for continual learning, consisting of novel theoretical insights and practical algorithmic designs. The research outcomes will substantially advance our understanding of continual learning, enrich its algorithmic framework, and provide scalable algorithm packages. The outcomes will be integrated into machine learning courses at all levels, benefiting students across disciplines such as electrical engineering, statistics, and computer science. The project will actively involve underrepresented students in STEM, synergizing research and education at undergraduate and graduate levels, and developing introductory materials for K-12 students through AI apprenticeship programs.The overarching goal of this project is to pioneer a methodological framework of continual learning for autonomous machine learners navigating dynamic data environments. The framework addresses three essential facets of continual learning. First, it will establish mathematical foundations of continual learning and scalable online algorithms to recollect an evolving memory for fast adaptation and capability expansion. Second, it will develop methods to amplify learning efficiency from informative, even less reliable, pseudo labels. Lastly, it will develop approaches for expanding the predictive power with task-specific assistance from peer learners who possess cross-modal data. The developed foundations will lead to novel algorithms to empower intelligent engineering systems to learn more efficiently and effectively in an ever-changing world. The project will also offer deep insights into pivotal machine learning questions, such as understanding the fundamental trade-offs between adaptivity and forgetting in dynamic learning environments, gauging the maximal influences of minor yet jointly dependent data perturbations on modeling results, and quantifying a model's room for improvement when the underlying law of probability remains elusive. Furthermore, these developments are anticipated to have broader impacts, enabling resource-constrained machine learners to continually broaden their learning capabilities in complex practical applications, such as intelligent transportation systems and cellular performance management.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.