Collaborative Research: PPoSS: Planning: Integrated Scalable Platform for Privacy-aware Collaborative Learning and Inference

用易于编程的软件构建未来可伸缩的分布式异构系统被广泛认为是一个巨大的挑战。人们普遍认为，随着处理器努力扩展并超越摩尔定律的终结，计算机系统的设计正在经历一场重大的颠覆。这种破坏体现在各种规模（片上、片上、节点上、机架上、集群上和数据中心上）的异构和分布式处理器和内存的新形式中，使可伸缩性成为所有级别的基本挑战。医疗保健分析为探索21世纪的可扩展系统设计提供了一个独特的机会，因为医疗机构在捕获和存储医疗数据，甚至实时数据流方面的能力已经发生了结构性转变。这一转变已经促成了机器学习（ML）模型的生态系统，用于各种临床任务的训练。要构建能够基于分布式医疗保健数据开发和部署ML模型的系统，就需要一种新的分布式异构体系结构，这些数据必须通过隐私保护约束进行访问。此外，拟议的架构必须附带一个软件框架，该框架可以满足特定领域数据科学家开发和增强部署在其医院中的ML模型的需求。该规划资助项目正在探索构建未来集成可扩展分布式系统所需的基本原则，以便准备向PPoSS计划提交完整的提案。它使用医疗保健分析领域来激励和具体化研究议程，但本研究中开发的原则也应该适用于其他应用领域。探索的重点是展示一个集成平台，该平台跨越计算和存储的多个分布和异构级别，同时也遵守重要的隐私约束。虽然最近在医疗保健应用中使用ML的进展令人鼓舞，但目前的方法没有达到未来系统所需的并行性、异构性和分布程度，或者b)支持许多临床情况下所需的对流数据的软实时响应。这个项目的独创性可以在一个统一的软件/硬件堆栈中集成分布、异构性和隐私考虑，其中包括跨越隐私保护联合持续学习的自适应资源管理，各个站点ML模型的自动专业化，以及最适合特定临床任务的ML模型的自动选择，这些模型在不同的延迟和软实时约束下最大限度地提高准确性。该项目开发基础可伸缩性原则的端到端方法将通过出版物、教程和课程影响计算机科学的多个领域，从而使未来分布式异构系统中研究可伸缩性挑战的其他研究人员受益。将医疗保健分析作为一种驱动应用程序，通过展示如何将从多个数据源中提取的知识体现在可以在现场运行的推荐系统中，从而为医生提供时间关键的决策支持，有可能为社会带来重大利益。作为进一步的影响，该项目将有助于培养机器学习系统和机器学习医疗保健交叉领域的高素质人才（HQP），这两个新兴的跨学科社区目前正在相互独立发展。最后，本研究将利用pi机构的现有活动，有助于扩大未被充分代表的群体在计算机领域的参与。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Building scalable distributed heterogeneous systems of the future with easy-to-program software is broadly acknowledged to be a grand challenge. It is widely recognized that a major disruption is currently under way in the design of computer systems as processors strive to extend, and go beyond, the end-game of Moore’s Law. This disruption is manifest in new forms of heterogeneous and distributed processors and memories at all scales (on-chip, on-die, on-node, on-rack, on-cluster, and on-data-center), rendering scalability as a fundamental challenge at all levels. Healthcare analytics offers a unique opportunity to explore scalable system design for the 21st century because there has been a tectonic shift in the ability of medical institutions to capture and store medical data, and to even stream data in real time. This shift has already contributed to an ecosystem of Machine Learning (ML) models being trained for a variety of clinical tasks. A new distributed heterogeneous architecture is required to build systems that can develop and deploy ML models based on distributed healthcare data that must necessarily be accessed with privacy-preserving constraints. Further, the proposed architecture must be accompanied by a software framework that can address the needs of domain-specific data scientists to develop and augment ML models being deployed in their hospitals.This planning grant project is exploring the foundational principles necessary in building integrated scalable distributed systems of the future, so as to prepare for submitting a full proposal to the PPoSS program. It uses the domain of healthcare analytics to motivate and concretize the research agenda, but the principles developed in this research should be applicable to other application domains as well. The exploration focuses on demonstrating an integrated platform that spans multiple levels of distribution and heterogeneity of computation and storage, while also obeying important privacy constraints. While recent progress on the use of ML in healthcare applications has been encouraging, current approaches do not a) scale to the degrees of parallelism, heterogeneity, and distribution that will be required in future systems, or b) support the soft real-time responsiveness to streaming data that is needed in many clinical situations. The originality of this project can be seen in the integration of distribution, heterogeneity, and privacy considerations in a single unified software/hardware stack, which includes adaptive resource management that spans privacy-preserving federated continuous learning, automatic specialization of ML models at individual sites, and automatic selection of ML models best suited for specific clinical tasks that maximize accuracy subject to different latency and soft real-time constraints.This project’s end-to-end approach to develop foundational scalability principles will impact multiple areas of computer science through publications, tutorials and courses, thereby benefiting other researchers working on scalability challenges in future distributed heterogeneous systems. The use of healthcare analytics as a driving application has the potential to result in significant benefits to society, by demonstrating how knowledge distilled from multiple sources of data can be embodied in recommendation systems that can run onsite to provide time-critical decision support to physicians. As a further impact, the project will contribute to the training of Highly Qualified Personnel (HQP) at the intersection of Systems for ML and ML for Healthcare — two emerging inter-disciplinary communities that are currently growing independent of each other. Finally, this research will leverage existing activities at the PIs’ institutions that contribute to broadening participation of underrepresented groups in computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.