Efficient deep learning in resource constrained environments

资源受限环境下的高效深度学习

• 批准号: 2053098
• 金额: --
• 依托单位: University of Oxford
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2018
• 资助国家: 英国
• 起止时间: 2018 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2053098
• 关键词: Efficient; deep; learning; resource; constrained

This project falls within the EPSRC Information and Communication (ICT) research area.The project "Efficient deep learning in resource constrained environments" aims to make further advances in accelerating and reducing memory footprint of machine learning (ML) models and, in particular, deep neural networks. The slowness of neural networks, even when run on expensive hardware, is commonly raised as an issue in both academia and industry. ML practitioners would like to deploy powerful models to low-powered hardware, such as mobile devices, and ML researchers would like to iterate quickly during model development, making it more interactive and productive.An ML system typically has the following components:(a) the hardware/software implementation which defines how the target device runs the model,(b) the model which defines how input data is processed (often depends on the structure of the data), and(c) the algorithm (loss function optimizer) which dictates how the model is trained on the input data.These components are largely seen as independent, enabling model developers to mix and match different options for their problem. As a result, existing research in model performance optimization tends to focus on each part in isolation.The aim of this project is to develop methods that extract further speed and memory gains by bypassing boundaries between said components, and applying optimisations throughout the entire ML system in a joint and mutually-informed way. For example, current software toolkits and hardware accelerators for deep neural networks often make little or no assumptions about the overall architecture of the neural network being ran or the structure of the data it is processing. Similarly, model architects tend to make little or no assumptions about the underlying hardware, missing out on potential speed or memory gains.Concretely, the objective of the project is to develop neural network implementation, design and training methodology that make the best use of capabilities and constraints of the target device as well as the structure of the data for the problem at hand. This would be of great use for industrial applications and fields of science which have to deal with vast quantities of data under time pressure or operate in environments restricted hardware, such as genomics, high energy physics, financial data analysis, etc.One of more ambitious research outcomes may also be a machine learning algorithm to discover such optimisations automatically when provided with a sample of the data and hardware constraints.The project aligns with the Artificial Intelligence EPSRC ICT research direction; no industrial collaborations are planned.

该项目福尔斯属于EPSRC信息和通信（ICT）研究领域，项目“资源受限环境中的高效深度学习”旨在进一步推进机器学习（ML）模型，特别是深度神经网络的加速和减少内存占用。即使在昂贵的硬件上运行，神经网络的速度也很慢，这在学术界和工业界都是一个普遍的问题。ML从业者希望将强大的模型部署到低功耗的硬件上，例如移动的设备，ML研究人员希望在模型开发过程中快速部署，使其更具交互性和生产力。ML系统通常具有以下组件：（a）定义目标设备如何运行模型的硬件/软件实现，（B）定义如何处理输入数据的模型（通常取决于数据的结构），以及（c）算法（损失函数优化器），它规定了如何在输入数据上训练模型。这些组件在很大程度上被视为独立的，使模型开发人员能够混合和匹配不同的选项来解决问题。因此，现有的模型性能优化研究往往集中在孤立的每个部分。该项目的目的是开发方法，通过绕过所述组件之间的边界，并以联合和相互通知的方式在整个ML系统中应用优化，来进一步提高速度和内存增益。例如，目前用于深度神经网络的软件工具包和硬件加速器通常很少或根本不假设正在运行的神经网络的整体架构或正在处理的数据结构。类似地，模型架构师倾向于对底层硬件做很少或根本不做假设，从而错过了潜在的速度或内存增益。具体来说，该项目的目标是开发神经网络实现，设计和训练方法，以便最好地利用目标设备的功能和约束以及手头问题的数据结构。这对于必须在时间压力下处理大量数据或在硬件受限的环境中操作的工业应用和科学领域（例如基因组学、高能物理学、金融数据分析、更雄心勃勃的研究成果之一也可以是机器学习算法，以在被提供有数据和硬件约束的样本时自动发现这样的优化。该项目与人工智能EPSRC ICT研究方向一致;没有计划进行工业合作。