CAREER: Automated and Efficient Machine Learning as a Service

机器学习即服务(MLaaS)是一种新兴的计算范式，它在云基础设施上提供机器学习任务的优化执行，如模型设计、模型训练和模型服务。模型复杂性和数据量的爆炸性增长，以及MLaaS需求的激增，已经导致计算资源和能源需求的大幅增加。遗憾的是，现有的MLaaS系统资源管理不善，对用户指定的性能和成本要求的支持有限，加剧了计算资源和能源的浪费。该项目旨在利用MLaaS的独特功能来设计高效、自动化和以用户为中心的MLaaS系统。该方法将通过各种新的优化方法，通过剔除不能满足模型服务延迟和目标精度的候选模型，显着减少资源浪费，缩短模型设计周期。为了支持完整的MLaaS工作流程，该项目还将开发MLaaS模型服务方法，使用智能自动伸缩以最小的资源消耗满足服务级别的延迟要求。该项目具有极大地降低资源和能源消耗以及与快速增长的社会对机器学习和云计算的需求相关的碳足迹的潜力。针对下一代机器学习系统和云基础设施的资源管理和节能，将产生重要的见解和技术。该项目的成果还将有助于并行和分布式系统、性能评估和优化以及绿色计算等相关领域。该项目将开展实质性的整合教育活动，包括新课程和在线教育开发，整合行业反馈在教育中的应用。此外，这项工作将通过结合最新的机器学习领域知识对本科生和研究生进行系统优化艺术方面的培训，同时结合来自代表性不足群体的学生，特别是女性的拓展和参与来影响他们。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine-Learning-as-a-Service (MLaaS) is an emerging computing paradigm that provides optimized execution of machine learning tasks, such as model design, model training, and model serving, on cloud infrastructure. Explosive growth in model complexity and data size along with the surging demands of MLaaS is already resulting in substantial increases in computational resource and energy requirements. Unfortunately, existing MLaaS systems have poor resource management and limited support for user specified performance and cost requirements, exacerbating waste in computing resources and energy. This project aims to utilize the unique features of MLaaS to design efficient, automated, and user-centric MLaaS systems. This approach will significantly reduce resource waste and shorten the model design cycles through a variety of novel optimization approaches and by eliminating candidate models that fail to meet model serving latency and target accuracy. To support complete MLaaS workflow, this project will also develop MLaaS model serving methodologies that can meet service level latency requirements with minimum resource consumption using intelligent autoscaling.This project has the potential to tremendously reduce the resource and energy consumptions as well as the carbon footprint associated with the fast-growing societal demands in machine learning and cloud computing. Important insights and technologies will be produced targeting resource management and energy saving of the next-generation machine learning systems and cloud infrastructure. The findings of this project will also contribute to related fields of parallel and distributed systems, performance evaluation and optimization, and green computing. This project will carry out substantial integrated education activities including new course and online education development, integration of industry feedback in education. Additionally, the work will impact undergraduate and graduate students by training them in the art of system optimization combined with the latest machine learning domain knowledge while combining outreach and engagement of students from underrepresented groups and especially women.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.