权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CRII: CSR: Mitigating Tail Latency in Prediction-serving Systems using Coding-theoretic Tools

CRII：CSR：使用编码理论工具减轻预测服务系统中的尾部延迟

基本信息

批准号：
1850483
负责人：
Rashmi Vinayak
金额：
$ 17.5万
依托单位：
Carnegie-Mellon University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-03-01 至 2020-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1850483&HistoricalAwards=false
关键词：
CRII CSR Mitigating Tail Latency

项目摘要

Prediction-serving systems take in queries and return predictions from a machine learning model. For example, an image classification service would take an image as a query and respond with a prediction of whether there is a car in the image. Prediction-serving systems are increasingly important for a wide variety of applications today. These systems are typically run in large-scale computing infrastructures, where failures and slowdowns are common. Such unavailability results in significant increase in the query-response latency, thereby degrading the quality-of-service. The research project designs and implements a resource-efficient solution for robustness in prediction-serving systems using a tool from the domain of coding theory called "erasure-coded computation".The overarching goal of the project is to design and implement a solution for reducing the potential long latency in prediction-serving systems using erasure-coded computations. The project overcomes critical limitations in existing coded-computation solutions by employing machine learning in a novel way. Specifically, the project involves the following key tasks: (1) Designing a low-overhead coded-computation solution that efficiently supports complex non-linear functions by employing a learning-based approach; (2) Designing an efficient training methodology conforming to the constraints of real-world prediction-serving systems; and (3) System design and implementation.The software resulting from the project will be integrated into open-source prediction-serving systems for use in both research and education. Active engagement in promoting diversity in science, technology, engineering and mathematics disciplines and undergraduate mentorship will be continued. The findings from the project will be integrated into graduate and undergraduate courses in computer science.The results from the research project, including software, will be made available at http://www.cs.cmu.edu/~rvinayak/crii .This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

预测服务系统接受查询并从机器学习模型返回预测。例如，图像分类服务会将图像作为查询，并预测图像中是否有汽车作为响应。预测服务系统对于当今的各种应用越来越重要。这些系统通常在大规模计算基础设施中运行，其中故障和速度缓慢很常见。这种不可用性会导致查询响应延迟显着增加，从而降低服务质量。该研究项目使用编码理论领域称为“纠删码计算”的工具，设计并实现了一种资源高效的解决方案，以实现预测服务系统的鲁棒性。该项目的总体目标是设计和实现一种解决方案，以减少使用纠删码计算的预测服务系统中潜在的长延迟。该项目通过以新颖的方式采用机器学习，克服了现有编码计算解决方案的关键限制。具体来说，该项目涉及以下关键任务：（1）设计一种低开销的编码计算解决方案，通过采用基于学习的方法有效支持复杂的非线性函数；（2）设计符合现实世界预测服务系统约束的高效训练方法； (3) 系统设计和实现。该项目产生的软件将被集成到开源预测服务系统中，用于研究和教育。将继续积极参与促进科学、技术、工程和数学学科的多样性以及本科生导师制。该项目的研究结果将被纳入计算机科学研究生和本科生课程中。该研究项目的结果（包括软件）将在 http://www.cs.cmu.edu/~rvinayak/crii 上提供。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。