CAREER: Towards Continual Learning on Evolving Graphs: from Memorization to Generalization

在现代大数据时代，数据往往不断增长，其互联互通和时间动力学不断演变。为了应对数据的持续演化，智能代理需要在其整个生命周期内增量地获取、感知、积累和利用结构和时间动态知识。这个项目旨在开发一个通用的机器学习范例来进行进化图的持续学习(COLEG)。该项目的成功将1)使关键基础设施(如社交网络、交通运输和可再生能源)和人类福利(例如，医疗保健和流行病学的改善)受益；2)为图表表示学习、时间序列分析、持续学习和因果分析领域的组成提供一个理想的平台；3)开发用于发展图表的开源工具，这些工具可以推进不同的主题，如节点分类、链接预测和时间预测，提高我们对物理世界的知识，并有助于现实世界的应用。该项目还将1)让高中生参与研究和接触K-12教师和学生，2)扩大未被充分代表的群体，特别是女性和低收入学生在STEM的参与，以及3)通过开发数据挖掘和机器学习方面的新课程模块来教育本科生和研究生。前者的目的是促进该模式获得全面保留大量可想象的情景的能力，包括现有结构和时间动态条件的全面范围。后者努力确保该模型能够概括其知识，并有效地适应不可预见和复杂的情况。这个项目将开发一个通用的机器学习范例，COLEG，通过保留基本的结构信息和时间动力学来解决灾难性的遗忘问题，确保泛化能力，并解决现实世界中关于演化图的应用。具体地说，1)将通过图稀疏和拓扑感知嵌入来制定新的持续学习范式来解决结构演化图中的灾难性遗忘问题，2)将开发新的算法来结合不同机制下演化图的结构和时间动态模式，解决无任务挑战，并揭示高阶依赖关系，3)将构建新的解决方案来追求预先训练的模型和促进时间测试自适应，以确保在不可预见的场景下的泛化。该项目将有助于持续学习、图形表示学习、时间序列分析和领域概括。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In the modern big data era, data often grows continuously and its interconnections and temporal dynamics evolve. To cope with the continuous evolution in data, an intelligent agent needs to incrementally acquire, perceive, accumulate, and exploit structural and temporal dynamic knowledge throughout its lifetime. This project aims to develop a generic machine learning paradigm to conduct Continual Learning on Evolving Graphs (CoLEG). The success of this project will 1) benefit critical infrastructure (such as social networks, transportation, and renewable energy) and human welfare (in the form of, for example, improvements in healthcare and epidemiology), 2) provide an ideal platform for composing the areas of graph representation learning, time series analysis, continual learning, and causal analysis, and 3) develop open-source tools for evolving graphs that can advance diverse topics such as node classification, link prediction, and temporal forecasting, improve our knowledge of the physical world, and contribute to real-world applications. This project will also 1) engage high school students in research and outreach to K-12 teachers and students, 2) broaden the participation of underrepresented groups especially female and low-income students in STEM, and 3) educate undergraduate and graduate students through the development of new course modules in data mining and machine learning.To achieve the above goals, it is imperative to address two core challenges associated with evolving graphs, specifically memorization and generalization. The former aims to facilitate the model in acquiring the capacity to comprehensively retain a vast array of conceivable scenarios, encompassing a comprehensive range of existing structural and temporal dynamic conditions. The latter strives to ensure the model can generalize its knowledge and effectively adapt to unforeseen and complex circumstances. This project will develop a generic machine learning paradigm, CoLEG, to resolve the catastrophic forgetting problem by retaining essential structural information and temporal dynamics, ensure the generalization capability, and address real-world applications on evolving graphs. Specifically, 1) a new continual learning paradigm will be formulated to tackle the catastrophic forgetting issue in structural evolving graphs via graph sparsification and topology-aware embedding, 2) new algorithms will be developed to incorporate structural and temporal dynamic patterns of evolving graphs under different regimes, resolve the task-free challenge, and reveal high-order dependencies, and 3) novel solutions will be constructed to pursue pre-trained models and facilitate test-of-time adaptation to ensure the generalization over unforeseen scenarios. This project will contribute to continual learning, graph representation learning, time series analysis, and domain generalization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.