CSR: Small: Deconstructing Distributed Deep Learning

CSR：小：解构分布式深度学习

• 批准号: 1816887
• 负责人: Leana Golubchik
• 金额: $ 51.6万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1816887&HistoricalAwards=false
• 关键词: CSR; Small; Deconstructing; Distributed; Deep

Deep learning has made substantial strides in computer vision, speech recognition, natural language processing, and other applications. New algorithms, larger datasets, increased compute power, and machine learning frameworks all contribute to this success. An important missing piece is that it is still challenging for users to effectively provision and integrate deep learning applications into existing datacenters. This project develops novel solutions that enable effective use of cloud resources, which in turn will aid in broadening the population of users capable of discovering new and better deep learning models and applying them in novel settings and applications.When developing new applications, users experiment with many deep neural networks (DNNs), but have limited knowledge of their computational demand. Due to non-linear scaling, predicting throughput improvements is challenging. Techniques developed in thrust 1 of this project quickly guide provisioning and resource allocation. In such environments, efficient inter-job resource sharing (particularly for similar DNNs) is an open problem, addressed in thrust 2 of the project by developing effective scheduling techniques. The diversity of datacenter workloads (DNNs, web), with different resource "affinity", creates opportunities to embrace cloud federations. While promising, there is a lack of techniques to support their sustainable deployment; these are developed in thrust 3 of the project.This project is committed to diversity in research and education, involving undergraduate and graduate students, coupled with an existing extensive K-12 outreach effort. The developed experimental testbed is utilized for both, research and education. All algorithms, designs, software, and data are made publicly available so that researchers and educators are able to replicate and improve on developed technologies. Solutions to the fundamental problems that are the focus of this project enable the development of new deep learning models and increase the adoption rate of these technologies in novel application domains.All reports and code are stored in an SVN-based repository. Software and related documents are publicly available on GitHub. All data is kept for at least 7 years beyond the life of the project. Research products are available promptly after publication, including supplemental information, through http://qed.usc.edu. These records are durable, accessible through standard web protocols, and made secure. Appropriate storage media is used, to keep data access current, as needed. Data that supports patents resulting from the project is retained for the duration of the patents. The URL to the repository is http://qed.usc.edu/D3/repository.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习在计算机视觉、语音识别、自然语言处理和其他应用方面取得了长足的进步。新算法、更大的数据集、更强的计算能力和机器学习框架都有助于这一成功。一个重要的缺失是，用户仍然很难有效地将深度学习应用程序集成到现有的数据中心中。该项目开发了能够有效利用云资源的新型解决方案，这反过来将有助于扩大能够发现新的和更好的深度学习模型并将其应用于新的设置和应用的用户群。在开发新应用时，用户尝试了许多深度神经网络（DNN），但对其计算需求的了解有限。由于非线性缩放，预测吞吐量改进是具有挑战性的。在本项目的第一个重点中开发的技术可以快速指导供应和资源分配。在这样的环境中，有效的作业间资源共享（特别是对于类似的DNN）是一个开放的问题，通过开发有效的调度技术，在项目的第二个目标中解决了这个问题。数据中心工作负载（DNN、Web）的多样性，具有不同的资源“亲和力”，为拥抱云联合创造了机会。虽然前景看好，但缺乏支持其可持续部署的技术;这些技术是在该项目的重点3中开发的，该项目致力于研究和教育的多样性，涉及本科生和研究生，加上现有的广泛的K-12推广工作。开发的实验测试平台用于研究和教育。所有的算法、设计、软件和数据都是公开的，这样研究人员和教育工作者就能够复制和改进开发的技术。该项目的重点是解决基本问题，这些解决方案可以开发新的深度学习模型，并提高这些技术在新应用领域的采用率。所有报告和代码都存储在基于SVN的存储库中。软件和相关文档可在GitHub上公开获取。所有数据将在项目生命周期结束后至少保存7年。研究产品在出版后立即可通过http://qed.usc.edu获得，包括补充信息。这些记录是持久的，可以通过标准的网络协议访问，并且是安全的。根据需要，使用适当的存储介质以保持数据访问最新。支持项目产生的专利的数据在专利有效期内保留。资源库的URL是http://qed.usc.edu/D3/repository.This奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Are Federated Cloud Sharing Systems Sustainable?: On Dynamic Sharing Markets and Their Stability

联合云共享系统可持续吗？：论动态共享市场及其稳定性

• DOI: 10.1109/tsusc.2019.2955093
• 发表时间: 2020
• 期刊: IEEE Transactions on Sustainable Computing
• 影响因子: 3.9
• 作者: Pal, Ranjan;Lin, Sung-Han;Ahuja, Aditya;Jagadeesan, Nachikethas;Kumar, Abhishek;Golubchik, Leana
• 通讯作者: Golubchik, Leana

Defending against Poisoning Backdoor Attacks on Federated Meta-learning

• DOI: 10.1145/3523062
• 发表时间: 2022-09
• 期刊: ACM Transactions on Intelligent Systems and Technology (TIST)
• 作者: Chien-Lun Chen;Sara Babakniya;Marco Paolieri;L. Golubchik

Performance and Revenue Analysis of Hybrid Cloud Federations with QoS Requirements

具有 QoS 要求的混合云联合的性能和收入分析

• DOI: 10.1109/cloud55607.2022.00055
• 发表时间: 2022
• 期刊: IEEE Cloud 2022
• 作者: B. Song, M. Paolieri

Predicting Inference Latency of Neural Architectures on Mobile Devices

• DOI: 10.1145/3578244.3583735
• 发表时间: 2023-04
• 期刊: Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering
• 作者: Zhuojin Li;Marco Paolieri;L. Golubchik

Achieving Transparency Report Privacy in Linear Time

在线性时间内实现透明度报告隐私

• DOI: 10.1145/3460001
• 发表时间: 2022
• 期刊: Journal of Data and Information Quality
• 作者: Chen, Chien-Lun;Golubchik, Leana;Pal, Ranjan
• 通讯作者: Pal, Ranjan