Leveraging FPGAs for Machine Learning Implementation and Acceleration

利用 FPGA 实现机器学习和加速

• 批准号: RGPIN-2020-07118
• 负责人: Brown, Stephen
• 金额: $ 2.84万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=741606
• 关键词: Leveraging; FPGAs; Machine; Learning; Implementation

This research program proposes to develop new technology that will lead to improved object detection and computer vision for autonomous vehicles using machine learning. The technology will include both software and hardware for compute-acceleration using field-programmable gate arrays (FPGAs). FPGAs are a type of integrated circuit chip that can be programmed to implement different applications in hardware. The program comprises three related research thrusts: neural-network (NN) algorithms that incorporate sensor-fusion techniques, NN circuit architectures for FPGAs, and end-user applications of this technology, such as enhancing vision for auto-drivers in conditions that are difficult for those drivers to manage. The traditional approach to object detection for autonomous vehicles has been a modified image classification task, which locates objects (e.g. cars, pedestrians) in a frame as well as performing an object classification task. However, as research has progressed, promising results with Light Detection and Ranging (LiDAR) data has catalyzed research into fusion of LiDAR and vision (camera) sensors. In this research program we propose to explore and fully understand fusion operations like the element-wise mean, but taking into consideration the predicted performance of individual sensors in novel ways. The performance evaluation of sensors becomes useful when one sensor is compromised through weather, component breakdown or intrusion attempts. When a sensor is incorrectly making predictions, the fusion-prediction network must be able to dynamically compensate to avoid degradation of the entire object detection system. State-of-the-art computer vision solutions for autonomous vehicles increasing rely on convolutional neural networks (CNNs) to achieve good quality of result. A CNN typically has millions of neurons and synapses which incur high computational complexity and storage requirements. Therefore, deploying CNNs on autonomous cars and drones as object detectors and sensor fusion solutions is difficult because of the tight power budget, low latency requirement, and scarce computation resources in these platforms. FPGAs have emerged as a popular substrate for implementing dedicated CNN accelerators for these applications. Results from this research program will be applied to a multi-university international self-driving automotive competition being sponsored by General Motors. Our focus will be on enhancing an automobile's ability to "see" properly in imperfect conditions, such as night driving, rain driving or snow driving. This technology could also be useful for any person who struggles with vision issues, including the elderly. Enhanced images could be provided through an existing wearable display, a clear display, via projection onto a windshield, and so on. Any person who suffers from vision restrictions could benefit greatly.

该研究计划旨在开发新技术，通过机器学习改进自动驾驶汽车的目标检测和计算机视觉。该技术将包括使用现场可编程门阵列（FPGA）进行计算加速的软件和硬件。FPGA是一种集成电路芯片，可以编程以实现硬件中的不同应用。该计划包括三个相关的研究重点：结合传感器融合技术的神经网络（NN）算法，FPGA的NN电路架构，以及该技术的最终用户应用，例如在驾驶员难以管理的条件下增强自动驾驶员的视觉。用于自主车辆的对象检测的传统方法是修改的图像分类任务，其定位帧中的对象（例如，汽车、行人）以及执行对象分类任务。然而，随着研究的进展，光探测和测距（LiDAR）数据的有希望的结果已经催化了对LiDAR和视觉（相机）传感器融合的研究。在这项研究计划中，我们建议探索和充分理解融合运算，如元素平均值，但要以新的方式考虑单个传感器的预测性能。当一个传感器因天气、组件故障或入侵企图而受到损害时，传感器的性能评估变得有用。当传感器错误地进行预测时，融合预测网络必须能够动态补偿，以避免整个目标检测系统的退化。 用于自动驾驶汽车的最先进的计算机视觉解决方案越来越依赖卷积神经网络（CNN）来实现良好的结果质量。CNN通常具有数百万个神经元和突触，这会导致高计算复杂性和存储要求。因此，在自主汽车和无人机上部署CNN作为对象检测器和传感器融合解决方案是困难的，因为这些平台的功率预算紧张，延迟要求低，计算资源稀缺。FPGA已经成为为这些应用实现专用CNN加速器的流行基板。该研究项目的结果将应用于通用汽车赞助的多大学国际自动驾驶汽车竞赛。我们的重点将是提高汽车在不完美的条件下正确“看到”的能力，例如夜间驾驶，雨天驾驶或雪地驾驶。这项技术也可能对任何有视力问题的人有用，包括老年人。增强图像可以通过现有的可穿戴显示器、清晰的显示器、投影到挡风玻璃上等来提供。任何视力受限的人都可以从中受益。