OAC Core: SHF: Small: Enabling Rapid Design and Deployment of Deep Learning Models on Hardware Accelerators

机器学习(ML)已迅速成为本世纪的基础技术之一。它在我们今天的生活中无处不在，从允许我们解锁智能手机，到为几乎任何人类活动(用餐、电影、服务等)提供推荐引擎。ML的应用预计在未来将变得更加具有变革性，特别是在医疗保健、自动运输、机器人、农业、教育和空间探索方面。ML模型的计算代价很高，需要大量内存来存储训练好的模型，并且有严格的运行时要求。它们不能在通用处理器上高效运行，这导致了ML定制硬件加速器的爆炸性增长。然而，从这些加速器中获得良好的性能和能效本身是具有挑战性的，因为它依赖于三个组件：ML模型本身、硬件参数以及将ML模型中的计算调度到加速器上有限的计算和内存资源上。拟议的研究将开发一个名为Maestro的开源软件网络基础设施，在实际构建硬件和部署模型之前，可以使用它来分析确定ML模型在目标硬件平台上的性能和能效。Maestro将对学生、研究人员和行业从业者学习、设计和部署定制的ML解决方案非常有用。该项目还将邀请本科生和高中生通过涉及黑客马拉松和硬件构建的外展活动来教授他们关于ML的知识。在加速器内的有限计算元素上映射ML计算，并理解需要在内存层次结构中移动的相应数据是一个很重要的问题；所有可能的切片和切分模型的方法(称为“数据流”)的空间都是指数级的复杂的，任何映射的好处在ML模型和目标加速器中都有所不同。为了解决这一问题，PI将首先开发一组以数据为中心的指令，以直接描述ML模型在加速器上的映射，这将允许精确计算跨空间和时间的数据重用机会，以减少总体数据移动。接下来，PI将开发大师分析成本模型框架，以评估目标硬件上的重用、端到端性能和能量。最后，将围绕Maestro开发一组工具，以自动搜索和确定给定运行时、功率、能量或面积限制的最佳硬件/映射/模型。所提出的框架将实现跨ML模型、映射和目标硬件的迭代创新和协同设计，因此对ML模型开发人员、编译器编写人员和计算机架构师具有很高的价值。Maestro将在开源许可下发布和维护，PI将定期运行教程，以在研究社区建立活跃的用户基础。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine Learning (ML) has rapidly emerged as one of the foundational technologies of this century. It is pervasive in our lives today, from allowing us to unlock smartphones, to powering recommendation engines for almost any human activities (dinning, movies, services etc). The applications of ML are expected to become even more transformative in the future, especially in healthcare, autonomous transport, robotics, agriculture, education, and space exploration. ML models are computationally expensive, need large amounts of memory to store the trained model, and have strict runtime requirements. They cannot be run efficiently on general-purpose processors, which has led to an explosive growth in custom hardware accelerators for ML. However, getting good performance and energy-efficiency from these accelerators is itself challenging, as it relies on three components: the ML model itself, the hardware parameters, and the scheduling of computations in the ML model onto the limited compute and memory resources on the accelerator. The proposed research will develop an open-source software cyberinfrastructure called MAESTRO that can be used to analytically determine the performance and energy-efficiency of ML models over target hardware platforms, prior to actually building the hardware and deploying the model. MAESTRO will be extremely useful for students, researchers, and industry practitioners alike to learn about, design, and deploy custom ML solutions. The project will also engage undergraduate and high-school students to teach them about ML through outreach activities involving hackathons and hardware building.Mapping ML computations over finite compute elements within an accelerator, and understanding the corresponding data that needs to move across the memory hierarchy is a non-trivial problem; the space of all possible ways of slicing and dicing the model (known as "dataflow") is exponentially complex, and the benefits of any mapping vary across ML models and target accelerator. To address this, the PI will first develop a set of data-centric directives to directly describe the mapping of the ML model over the accelerator, which will enable precise calculations of data reuse opportunities across space and time to reduce overall data movement. Next, the PI will develop the MAESTRO analytical cost model framework to estimate reuse, end-to-end performance, and energy over the target hardware. Finally, a set of tools will be developed around MAESTRO to automatically search for and determine the optimal hardware/mapping/model given constraints of runtime, power, energy, or area. The proposed framework will enable iterative innovation and co-design across the ML model, mapping and target hardware, and will therefore be highly valuable for ML model developers, compiler writers and computer architects. MAESTRO will be released and maintained on an open-source license, and the PI will run periodic tutorials to build an active user-base in the research community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.