CAREER: Towards Continual Learning on Evolving Graphs: from Memorization to Generalization

在现代大数据时代，数据经常不断增长，其相互联系和时间动态也在不断发展。为了科普数据的不断演变，智能代理需要在其整个生命周期中逐步获取，感知，积累和利用结构和时间动态知识。该项目旨在开发一个通用的机器学习范式，以进行不断学习的进化图（CoLEG）。该项目的成功将1）有利于关键基础设施（如社交网络、交通和可再生能源）和人类福利（以例如医疗保健和流行病学的改进的形式），2）提供用于构成图形表示学习、时间序列分析、持续学习和因果分析的领域的理想平台，3）开发用于进化图的开源工具，这些工具可以推进节点分类、链接预测和时间预测等各种主题，提高我们对物理世界的认识，并为现实世界的应用做出贡献。该项目还将1）让高中生参与研究并推广到K-12教师和学生，2）扩大代表性不足的群体，特别是女性和低收入学生在STEM中的参与，以及3）通过开发数据挖掘和机器学习的新课程模块来教育本科生和研究生。必须解决与不断发展的图表相关的两个核心挑战，即记忆和概括。前者的目的是促进模型获得全面保留大量可设想情景的能力，包括全面的现有结构和时间动态条件。后者努力确保模型能够概括其知识，并有效地适应不可预见的复杂情况。该项目将开发一个通用的机器学习范式CoLEG，通过保留基本的结构信息和时间动态来解决灾难性遗忘问题，确保泛化能力，并解决进化图上的现实应用。具体而言，1）将制定一个新的持续学习范式，通过图稀疏化和拓扑感知嵌入来解决结构演化图中的灾难性遗忘问题，2）将开发新的算法，以结合不同制度下演化图的结构和时间动态模式，解决无任务挑战，并揭示高阶依赖关系，以及3）将构造新的解决方案以追求预先训练的模型并促进时间适应性测试，以确保在不可预见的场景下的泛化。该项目将有助于持续学习、图形表示学习、时间序列分析和领域概括。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

In the modern big data era, data often grows continuously and its interconnections and temporal dynamics evolve. To cope with the continuous evolution in data, an intelligent agent needs to incrementally acquire, perceive, accumulate, and exploit structural and temporal dynamic knowledge throughout its lifetime. This project aims to develop a generic machine learning paradigm to conduct Continual Learning on Evolving Graphs (CoLEG). The success of this project will 1) benefit critical infrastructure (such as social networks, transportation, and renewable energy) and human welfare (in the form of, for example, improvements in healthcare and epidemiology), 2) provide an ideal platform for composing the areas of graph representation learning, time series analysis, continual learning, and causal analysis, and 3) develop open-source tools for evolving graphs that can advance diverse topics such as node classification, link prediction, and temporal forecasting, improve our knowledge of the physical world, and contribute to real-world applications. This project will also 1) engage high school students in research and outreach to K-12 teachers and students, 2) broaden the participation of underrepresented groups especially female and low-income students in STEM, and 3) educate undergraduate and graduate students through the development of new course modules in data mining and machine learning.To achieve the above goals, it is imperative to address two core challenges associated with evolving graphs, specifically memorization and generalization. The former aims to facilitate the model in acquiring the capacity to comprehensively retain a vast array of conceivable scenarios, encompassing a comprehensive range of existing structural and temporal dynamic conditions. The latter strives to ensure the model can generalize its knowledge and effectively adapt to unforeseen and complex circumstances. This project will develop a generic machine learning paradigm, CoLEG, to resolve the catastrophic forgetting problem by retaining essential structural information and temporal dynamics, ensure the generalization capability, and address real-world applications on evolving graphs. Specifically, 1) a new continual learning paradigm will be formulated to tackle the catastrophic forgetting issue in structural evolving graphs via graph sparsification and topology-aware embedding, 2) new algorithms will be developed to incorporate structural and temporal dynamic patterns of evolving graphs under different regimes, resolve the task-free challenge, and reveal high-order dependencies, and 3) novel solutions will be constructed to pursue pre-trained models and facilitate test-of-time adaptation to ensure the generalization over unforeseen scenarios. This project will contribute to continual learning, graph representation learning, time series analysis, and domain generalization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.