Collaborative Research: PPoSS: Planning: Efficient and Scalable Learning and Management of Distributed Probabilistic Graphs

云计算基础设施和机器学习算法的进步，推动了从自动化药物设计到自然语言理解等各个领域的变革技术。然而，理解完整的软件/硬件堆栈对于开发可扩展的特定领域机器学习模型的领域专家来说仍然是一个巨大的挑战，特别是当应用程序数据具有固有的非关系表示时。该项目的新颖之处在于探索、设计和实现一个端到端系统，该系统提供了高效和有效的概率图管理，可以作为各种领域（例如，社交网络、生物信息学、传感和通信）的一般数据抽象。概率图模型不仅捕获了现实世界实体之间复杂的相关性，而且还量化了它们之间的相关性或影响的强度。该项目的影响在于，它解决了理论和系统实践中重要的缺失部分，以系统的、归纳的和可验证的方式支持概率图管理。这个计划授权项目研究了一个端到端的概率图管理系统，该系统承诺在可扩展的分布式设置中高效的概率图学习、表示、聚合和分析，并提供质量保证。重点探讨了概率图管理的完整软件/硬件堆栈，包括设计形式化的概率图定义/操作抽象，以及可证明的归纳约束编译过程，保证分布式环境下流水线执行的正确性和效率。这个计算框架可以作为一个通用的概率图分析工具，通过以更全面和变革的方式发现和理解现实世界实体之间的复杂相关性，使不同的研究领域受益。除了这一优势之外，该项目的成果，如开源软件、出版物和讲习班教程，可以有利于数据管理研究、行业总体决策处理（基于传感的自动操作，例如自动驾驶、自动驾驶）和政府（数据驱动的政策制定，例如公共卫生/全球贸易监测）。此外，该项目的产品可以整合到丰富本科/研究生水平课程的课程开发中（包括与云计算、数据管理和机器学习相关的课程项目），从而在研究机构中培训/受益于大量代表性不足的学生（包括少数民族/女学生）。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The advancement of cloud-computing infrastructure and machine-learning algorithms have enabled transformative techniques that push the boundaries of various domains, ranging from automated drug design to natural-language understanding. However, understanding the full software/hardware stack remains a grand challenge for domain experts in developing scalable domain-specific machine-learning models, especially when the application data is of inherently non-relational representations. The project’s novelties are to explore, design, and implement an end-to-end system that delivers efficient and effective management of probabilistic graphs, which can serve as a general data abstraction in a variety of domains (e.g., social network, bioinformatics, sensing and communication, to name a few). The probabilistic graph model not only captures complicated correlations among real-world entities but also quantifies the intensities of correlations or influences among them. The project’s impacts are that it addresses important missing pieces from both theory and system practices to support probabilistic graph management in a systematic, inductive, and verifiable way. This planning-grant project investigates an end-to-end probabilistic graph management system that promises efficient probabilistic graph learning, representation, aggregation, and analysis with quality guarantees in a scalable distributed setting. The exploration focuses on the full software/hardware stack of probabilistic-graph management, including designing formal probabilistic-graph definition/manipulation abstractions, and the provable compiling process of inductive constraints with guaranteed correctness and efficiency of pipelining execution in a distributed setting. This computing framework can serve as a general-purpose probabilistic-graph analysis tool that benefits different research domains by discovering and understanding the complex correlations among real-world entities in a more comprehensive and transformative way. Besides this advantage, the outcomes of this project, such as open-source software, publications, and workshop tutorials, could benefit data-management research, decision-making processing in general for the industry (sensing-based automatic operations, e.g., auto-piloting, self-driving), and the government (data-driven policymaking, e.g., public health/global trading monitoring). Furthermore, products from this project can be integrated to enrich the curriculum development of undergraduate/graduate-level courses (with course projects related to cloud computing, data management, and machine learning) and therefore train/benefit a rich body of underrepresented students (including minority/female students) at the investigators' institutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.