Leveraging FPGAs for Machine Learning Implementation and Acceleration

点击翻译按钮获取中文摘要

This research program proposes to develop new technology that will lead to improved object detection and computer vision for autonomous vehicles using machine learning. The technology will include both software and hardware for compute-acceleration using field-programmable gate arrays (FPGAs). FPGAs are a type of integrated circuit chip that can be programmed to implement different applications in hardware. The program comprises three related research thrusts: neural-network (NN) algorithms that incorporate sensor-fusion techniques, NN circuit architectures for FPGAs, and end-user applications of this technology, such as enhancing vision for auto-drivers in conditions that are difficult for those drivers to manage. The traditional approach to object detection for autonomous vehicles has been a modified image classification task, which locates objects (e.g. cars, pedestrians) in a frame as well as performing an object classification task. However, as research has progressed, promising results with Light Detection and Ranging (LiDAR) data has catalyzed research into fusion of LiDAR and vision (camera) sensors. In this research program we propose to explore and fully understand fusion operations like the element-wise mean, but taking into consideration the predicted performance of individual sensors in novel ways. The performance evaluation of sensors becomes useful when one sensor is compromised through weather, component breakdown or intrusion attempts. When a sensor is incorrectly making predictions, the fusion-prediction network must be able to dynamically compensate to avoid degradation of the entire object detection system. State-of-the-art computer vision solutions for autonomous vehicles increasing rely on convolutional neural networks (CNNs) to achieve good quality of result. A CNN typically has millions of neurons and synapses which incur high computational complexity and storage requirements. Therefore, deploying CNNs on autonomous cars and drones as object detectors and sensor fusion solutions is difficult because of the tight power budget, low latency requirement, and scarce computation resources in these platforms. FPGAs have emerged as a popular substrate for implementing dedicated CNN accelerators for these applications. Results from this research program will be applied to a multi-university international self-driving automotive competition being sponsored by General Motors. Our focus will be on enhancing an automobile's ability to "see" properly in imperfect conditions, such as night driving, rain driving or snow driving. This technology could also be useful for any person who struggles with vision issues, including the elderly. Enhanced images could be provided through an existing wearable display, a clear display, via projection onto a windshield, and so on. Any person who suffers from vision restrictions could benefit greatly.