Leveraging FPGAs for Machine Learning Implementation and Acceleration

该研究计划建议开发新技术，利用机器学习改进自动驾驶汽车的目标检测和计算机视觉。该技术将包括使用现场可编程门阵列（fpga）进行计算加速的软件和硬件。fpga是一种集成电路芯片，可以通过编程在硬件上实现不同的应用。该项目包括三个相关的研究重点：结合传感器融合技术的神经网络（NN）算法，用于fpga的神经网络电路架构，以及该技术的最终用户应用，例如在难以管理的条件下增强汽车驾驶员的视觉。自动驾驶汽车的传统目标检测方法是一种改进的图像分类任务，该任务定位帧中的目标（例如汽车，行人）并执行目标分类任务。然而，随着研究的进展，光探测和测距（LiDAR）数据的有希望的结果催化了激光雷达和视觉（相机）传感器融合的研究。在本研究计划中，我们建议探索和充分理解融合操作，如元素智能均值，但考虑到单个传感器的预测性能以新颖的方式。当一个传感器因天气、组件故障或入侵企图而受损时，传感器的性能评估变得有用。当传感器做出错误预测时，融合预测网络必须能够动态补偿，以避免整个目标检测系统的退化。最先进的自动驾驶汽车计算机视觉解决方案越来越依赖于卷积神经网络（cnn）来获得高质量的结果。CNN通常有数百万个神经元和突触，这带来了很高的计算复杂度和存储要求。因此，在自动驾驶汽车和无人机上部署cnn作为目标探测器和传感器融合解决方案是困难的，因为这些平台的功耗预算紧张，延迟要求低，计算资源稀缺。fpga已经成为为这些应用实现专用CNN加速器的流行基板。该研究项目的成果将应用于由通用汽车公司赞助的多所大学国际自动驾驶汽车竞赛。我们的重点将是提高汽车在不完美条件下的“视觉”能力，比如夜间驾驶、雨天驾驶或雪天驾驶。这项技术对任何有视力问题的人也很有用，包括老年人。增强图像可以通过现有的可穿戴显示器、清晰显示器、投影到挡风玻璃上等等来提供。任何患有视力障碍的人都可以从中受益匪浅。

This research program proposes to develop new technology that will lead to improved object detection and computer vision for autonomous vehicles using machine learning. The technology will include both software and hardware for compute-acceleration using field-programmable gate arrays (FPGAs). FPGAs are a type of integrated circuit chip that can be programmed to implement different applications in hardware. The program comprises three related research thrusts: neural-network (NN) algorithms that incorporate sensor-fusion techniques, NN circuit architectures for FPGAs, and end-user applications of this technology, such as enhancing vision for auto-drivers in conditions that are difficult for those drivers to manage. The traditional approach to object detection for autonomous vehicles has been a modified image classification task, which locates objects (e.g. cars, pedestrians) in a frame as well as performing an object classification task. However, as research has progressed, promising results with Light Detection and Ranging (LiDAR) data has catalyzed research into fusion of LiDAR and vision (camera) sensors. In this research program we propose to explore and fully understand fusion operations like the element-wise mean, but taking into consideration the predicted performance of individual sensors in novel ways. The performance evaluation of sensors becomes useful when one sensor is compromised through weather, component breakdown or intrusion attempts. When a sensor is incorrectly making predictions, the fusion-prediction network must be able to dynamically compensate to avoid degradation of the entire object detection system. State-of-the-art computer vision solutions for autonomous vehicles increasing rely on convolutional neural networks (CNNs) to achieve good quality of result. A CNN typically has millions of neurons and synapses which incur high computational complexity and storage requirements. Therefore, deploying CNNs on autonomous cars and drones as object detectors and sensor fusion solutions is difficult because of the tight power budget, low latency requirement, and scarce computation resources in these platforms. FPGAs have emerged as a popular substrate for implementing dedicated CNN accelerators for these applications. Results from this research program will be applied to a multi-university international self-driving automotive competition being sponsored by General Motors. Our focus will be on enhancing an automobile's ability to "see" properly in imperfect conditions, such as night driving, rain driving or snow driving. This technology could also be useful for any person who struggles with vision issues, including the elderly. Enhanced images could be provided through an existing wearable display, a clear display, via projection onto a windshield, and so on. Any person who suffers from vision restrictions could benefit greatly.