CSR: Small: Towards Efficient Deep Inference for Mobile Applications

越来越多的移动应用程序正在使用深度学习模型来提供新颖而有用的功能，如语言翻译和对象识别。通过将输入数据(例如照片或音频剪辑)传递给复杂模型以生成有意义的输出来支持这些功能。然而，使用深度学习模型的移动应用程序目前需要在开发时在预测精度和速度之间做出选择。这可能会由于在较旧的移动设备上运行最先进的型号等原因而导致用户体验不佳。提出的MODI(移动深度推理)项目概述了在设计和实现移动感知深度推理平台方面的新研究，该平台结合了算法和系统优化方面的创新。拟议的工作将通过实现灵活的细粒度模型划分和基于层的推理执行以及特定于移动的模型设计来解决移动深度推理的性能问题。此外，Modi支持可扩展的移动深度推理范例，并在设备上和云中进行高效的模型管理。该项目将使深度学习能够为移动应用程序提供有用的功能，并显著提高性能。因此，该项目将打开大门，允许在资源更有限的设备(如嵌入式设备)上运行优化的深度学习模型。Modi项目可以用作独立的云系统，也可以通过整合其特定于移动的优化功能与现有的通用推理服务平台集成，从而提高采用率。该项目的更广泛影响将包括纳入研究成果的研究生和本科课程、让本科生和K-12学生接触计算机系统研究和深度学习的外联活动。此外，与项目相关的源代码和其他资源将通过项目网站发布给研究社区，http://tianguo.info/projects/modi.htmlThis奖项反映了国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

An ever-increasing number of mobile applications are using deep learning models to provide novel and useful features, such as language translation and object recognition. These features are supported by passing input data, for example a photo or an audio clip, to complex models in order to generate meaningful output. However, mobile applications that use deep learning models currently need to choose between prediction accuracy and speed at development time. This can lead to poor user experience due to reasons such as running state-of-the-art models on older mobile devices. The proposed MODI (MObile Deep Inference) project outlines new research in designing and implementing a mobile-aware deep inference platform that combines innovations in both algorithm and system optimizations. The proposed work will address mobile deep inference performance problems by enabling flexible, fine-grained model partition and layer-based inference execution, as well as mobile-specific model designs. In addition, MODI enables a scalable mobile deep inference paradigm with efficient model management both on-device and in the cloud. The project will empower deep learning to provide useful features for mobile applications with significantly improved performance. Consequently, this project will open doors to allow running optimized deep learning models on much more resource-constrained devices such as embedded devices. The MODI project can be used as a standalone cloud system or integrated with existing general inference serving platforms by incorporating its mobile-specific optimizations, thereby increasing adoption. The broader impacts of the project will include graduate and undergraduate courses that incorporate research results, outreach to expose undergraduates and K-12 students to research in both computer systems and deep learning. In addition, project related source code and other resources will be released to the research community through the project website at http://tianguo.info/projects/modi.htmlThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.