Collaborative Research: PPoSS: Planning: Integrated Scalable Platform for Privacy-aware Collaborative Learning and Inference

用易于编程的软件构建未来可伸缩的分布式异构系统被广泛认为是一个巨大的挑战。人们普遍认识到，随着处理器努力扩展并超越摩尔定律的最终结果，计算机系统的设计中目前正在发生重大的颠覆。这种颠覆体现在各种规模（片上、管芯上、节点上、机架上、集群上和数据中心上）的异构和分布式处理器和存储器的新形式中，使可扩展性成为所有级别的基本挑战。 医疗保健分析为探索21世纪世纪的可扩展系统设计提供了一个独特的机会，因为医疗机构捕获和存储医疗数据的能力已经发生了结构性转变，甚至可以真实的实时传输数据。这一转变已经促成了一个机器学习（ML）模型的生态系统，该生态系统正在为各种临床任务进行训练。需要一个新的分布式异构架构来构建系统，该系统可以基于分布式医疗数据开发和部署ML模型，这些数据必须在隐私保护约束下访问。此外，建议的架构必须伴随着一个软件框架，可以解决特定领域的数据科学家的需求，以开发和增强ML模型正在部署在他们的hospital.This规划补助金项目正在探索必要的基本原则，在构建集成的可扩展的分布式系统的未来，以便准备提交一个完整的建议PPoSS计划。 它使用医疗保健分析领域来激励和具体化研究议程，但本研究中开发的原则也应该适用于其他应用领域。 探索的重点是展示一个集成的平台，跨越多个层次的分布和异构的计算和存储，同时也遵守重要的隐私约束。虽然最近在医疗保健应用中使用ML的进展令人鼓舞，但目前的方法并没有a）扩展到未来系统所需的并行度，异构性和分布性，或者B）支持许多临床情况下所需的流数据的软实时响应。该项目的独创性可以在单个统一的软件/硬件堆栈中集成分布，异构性和隐私考虑因素，其中包括跨越隐私保护联邦连续学习的自适应资源管理，单个站点的ML模型的自动专业化，自动选择最适合特定临床任务的ML模型，最大限度地提高不同延迟和软真实的准确性。这个项目的端到端的方法来开发基本的可扩展性原则将影响计算机科学的多个领域，通过出版物，教程和课程，从而有利于其他研究人员在未来的分布式异构系统的可扩展性的挑战。将医疗分析作为一种驱动应用，有可能为社会带来重大利益，因为它展示了如何将从多个数据源中提取的知识体现在推荐系统中，这些系统可以在现场运行，为医生提供时间关键的决策支持。作为进一步的影响，该项目将有助于在ML系统和ML医疗保健的交叉点培训高素质人员（HQP）-这两个新兴的跨学科社区目前正在相互独立。最后，这项研究将利用PI机构的现有活动，这些活动有助于扩大代表性不足的群体在计算中的参与。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Building scalable distributed heterogeneous systems of the future with easy-to-program software is broadly acknowledged to be a grand challenge. It is widely recognized that a major disruption is currently under way in the design of computer systems as processors strive to extend, and go beyond, the end-game of Moore’s Law. This disruption is manifest in new forms of heterogeneous and distributed processors and memories at all scales (on-chip, on-die, on-node, on-rack, on-cluster, and on-data-center), rendering scalability as a fundamental challenge at all levels. Healthcare analytics offers a unique opportunity to explore scalable system design for the 21st century because there has been a tectonic shift in the ability of medical institutions to capture and store medical data, and to even stream data in real time. This shift has already contributed to an ecosystem of Machine Learning (ML) models being trained for a variety of clinical tasks. A new distributed heterogeneous architecture is required to build systems that can develop and deploy ML models based on distributed healthcare data that must necessarily be accessed with privacy-preserving constraints. Further, the proposed architecture must be accompanied by a software framework that can address the needs of domain-specific data scientists to develop and augment ML models being deployed in their hospitals.This planning grant project is exploring the foundational principles necessary in building integrated scalable distributed systems of the future, so as to prepare for submitting a full proposal to the PPoSS program. It uses the domain of healthcare analytics to motivate and concretize the research agenda, but the principles developed in this research should be applicable to other application domains as well. The exploration focuses on demonstrating an integrated platform that spans multiple levels of distribution and heterogeneity of computation and storage, while also obeying important privacy constraints. While recent progress on the use of ML in healthcare applications has been encouraging, current approaches do not a) scale to the degrees of parallelism, heterogeneity, and distribution that will be required in future systems, or b) support the soft real-time responsiveness to streaming data that is needed in many clinical situations. The originality of this project can be seen in the integration of distribution, heterogeneity, and privacy considerations in a single unified software/hardware stack, which includes adaptive resource management that spans privacy-preserving federated continuous learning, automatic specialization of ML models at individual sites, and automatic selection of ML models best suited for specific clinical tasks that maximize accuracy subject to different latency and soft real-time constraints.This project’s end-to-end approach to develop foundational scalability principles will impact multiple areas of computer science through publications, tutorials and courses, thereby benefiting other researchers working on scalability challenges in future distributed heterogeneous systems. The use of healthcare analytics as a driving application has the potential to result in significant benefits to society, by demonstrating how knowledge distilled from multiple sources of data can be embodied in recommendation systems that can run onsite to provide time-critical decision support to physicians. As a further impact, the project will contribute to the training of Highly Qualified Personnel (HQP) at the intersection of Systems for ML and ML for Healthcare — two emerging inter-disciplinary communities that are currently growing independent of each other. Finally, this research will leverage existing activities at the PIs’ institutions that contribute to broadening participation of underrepresented groups in computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.