Collaborative Research: PPoSS: Planning: Efficient and Scalable Learning and Management of Distributed Probabilistic Graphs

云计算基础设施和机器学习算法的进步使变革性技术成为可能，这些技术推动了从自动化药物设计到自然语言理解等各个领域的边界。 然而，理解完整的软件/硬件堆栈仍然是领域专家在开发可扩展的特定领域机器学习模型时面临的巨大挑战，特别是当应用程序数据具有固有的非关系表示时。该项目的新颖之处在于探索、设计和实现一个端到端系统，该系统提供了对概率图的高效管理，可以作为各种领域（例如，社交网络、生物信息学、传感和通信，仅举几例）。概率图模型不仅捕捉了现实世界实体之间的复杂关联，而且还量化了它们之间的关联或影响的强度。该项目的影响是，它解决了理论和系统实践中的重要缺失部分，以系统的，归纳的和可验证的方式支持概率图管理。 这个计划资助项目研究了一个端到端的概率图管理系统，该系统承诺在可扩展的分布式环境中进行有效的概率图学习，表示，聚合和分析，并具有质量保证。探索的重点是完整的软件/硬件堆栈的概率图管理，包括设计正式的概率图定义/操作抽象，和可证明的编译过程中的归纳约束，保证正确性和效率的流水线执行在分布式设置。这种计算框架可以作为一种通用的概率图分析工具，通过以更全面和变革的方式发现和理解现实世界实体之间的复杂相关性，使不同的研究领域受益。 除了这一优势之外，该项目的成果，如开源软件、出版物和研讨会教程，还可以使数据管理研究、一般行业的决策处理（基于传感的自动操作，例如，自动驾驶、自动驾驶）和政府（数据驱动的决策，例如，公共卫生/全球贸易监测）。 此外，该项目的产品可以整合，以丰富本科/研究生课程的课程开发（与云计算，数据管理，和机器学习），从而培养/使大量代表性不足的学生受益（包括少数民族/女学生）该奖项反映了NSF的法定使命，并被认为是值得通过评估使用基金会的知识优点和更广泛的影响审查标准。

The advancement of cloud-computing infrastructure and machine-learning algorithms have enabled transformative techniques that push the boundaries of various domains, ranging from automated drug design to natural-language understanding. However, understanding the full software/hardware stack remains a grand challenge for domain experts in developing scalable domain-specific machine-learning models, especially when the application data is of inherently non-relational representations. The project’s novelties are to explore, design, and implement an end-to-end system that delivers efficient and effective management of probabilistic graphs, which can serve as a general data abstraction in a variety of domains (e.g., social network, bioinformatics, sensing and communication, to name a few). The probabilistic graph model not only captures complicated correlations among real-world entities but also quantifies the intensities of correlations or influences among them. The project’s impacts are that it addresses important missing pieces from both theory and system practices to support probabilistic graph management in a systematic, inductive, and verifiable way. This planning-grant project investigates an end-to-end probabilistic graph management system that promises efficient probabilistic graph learning, representation, aggregation, and analysis with quality guarantees in a scalable distributed setting. The exploration focuses on the full software/hardware stack of probabilistic-graph management, including designing formal probabilistic-graph definition/manipulation abstractions, and the provable compiling process of inductive constraints with guaranteed correctness and efficiency of pipelining execution in a distributed setting. This computing framework can serve as a general-purpose probabilistic-graph analysis tool that benefits different research domains by discovering and understanding the complex correlations among real-world entities in a more comprehensive and transformative way. Besides this advantage, the outcomes of this project, such as open-source software, publications, and workshop tutorials, could benefit data-management research, decision-making processing in general for the industry (sensing-based automatic operations, e.g., auto-piloting, self-driving), and the government (data-driven policymaking, e.g., public health/global trading monitoring). Furthermore, products from this project can be integrated to enrich the curriculum development of undergraduate/graduate-level courses (with course projects related to cloud computing, data management, and machine learning) and therefore train/benefit a rich body of underrepresented students (including minority/female students) at the investigators' institutions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.