XPS:FULL:DSD: Collaborative Research: FPGA Cloud Platform for Deep Learning, Applications in Computer Vision

XPS:FULL:DSD：协作研究：深度学习 FPGA 云平台、计算机视觉应用

• 批准号: 1533739
• 负责人: Michael Ferdman
• 金额: $ 57.4万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1533739&HistoricalAwards=false
• 关键词: XPS; FULL; DSD; Collaborative; Research

We stand on the verge of dramatic advances in deep learning applications, which will soon enable practicality and widespread adoption of computer vision based recognition in scientific inquiry, commercial applications, and everyday life. Grand challenge problems are within our reach; we will soon be able to build automated systems that recognize nearly everything we see, systems that can recognize the tens of thousands of basic-level categories that psychologists posit humans can recognize, systems that continuously learn from photos, video, and web content in order to create more complete and accurate visual models of the world. However, while it is clear that the computational capabilities for deep learning are within reach, it is equally clear that the required computational power cannot come from general-purpose processors. To succeed, we will need to build specialized domain-specific computing systems based on hardware accelerators that are capable of exploiting the extreme fine-grained parallelism inherent in deep-learning workloads. This project leverages parallelization and reconfigurable hardware to create an automated system that distributes computer vision algorithms onto a large number of field-programmable gate arrays (FPGA Cloud). This project builds on recent advances in domain-specific hardware generation tools in order to bring the potential parallelism and performance per watt advantages of FPGAs to large-scale computer vision problems. By developing a platform to run deep learning algorithms on large clouds of FPGAs, this proposal explicitly addresses scaling algorithms beyond what a single chip can process. This involves addressing a wide range of challenging problems in algorithm analysis, building domain-specific hardware generators, communication for scaling algorithms across multiple FPGAs, and extensive validation of generating hardware for state-of-the-art deep learning approaches applied to computer vision problems. This project advances tools for designing domain-specific FPGA implementations of algorithms, taking a step toward making more efficient computing with greater parallelism more widely available. In particular, for computer vision, there will be significant benefits from a product of multiple improvements: higher parallelism, lower gate requirement by moving to fixed point when possible, and better performance per watt leading to higher computation density in servers. Together, these have the potential to significantly increase the extent to which computer vision can be a part of our daily lives, making computers better able to understand the context of our world.

我们站在深度学习应用的巨大进步的边缘，这将很快使基于计算机视觉的识别在科学研究、商业应用和日常生活中得到实用和广泛的采用。 我们很快就能构建出几乎能识别我们所看到的一切的自动化系统，能够识别心理学家所能识别的成千上万个基本类别的系统，能够从照片、视频和网络内容中不断学习的系统，以创建更完整、更准确的世界视觉模型。 然而，虽然深度学习的计算能力是可以实现的，但同样清楚的是，所需的计算能力无法来自通用处理器。 为了取得成功，我们需要基于硬件加速器构建专门的特定领域计算系统，这些硬件加速器能够利用深度学习工作负载中固有的极细粒度并行性。 该项目利用并行化和可重构硬件来创建一个自动化系统，将计算机视觉算法分发到大量现场可编程门阵列（FPGA Cloud）上。该项目建立在特定领域硬件生成工具的最新进展基础上，以便将FPGA的潜在并行性和每瓦性能优势带到大规模计算机视觉问题中。 通过开发一个在大型FPGA云上运行深度学习算法的平台，该提案明确地解决了扩展算法超出单个芯片可以处理的问题。 这涉及解决算法分析中的各种挑战性问题，构建特定领域的硬件生成器，跨多个FPGA扩展算法的通信，以及为应用于计算机视觉问题的最先进深度学习方法生成硬件的广泛验证。 该项目推进了用于设计算法的特定领域FPGA实现的工具，朝着更广泛地提供具有更大并行性的更高效计算迈出了一步。 特别是对于计算机视觉，多项改进的产品将带来显著的好处：更高的并行性，尽可能移动到固定点以降低门要求，以及更好的每瓦性能，从而提高服务器的计算密度。 总之，这些都有可能显着提高计算机视觉成为我们日常生活一部分的程度，使计算机能够更好地理解我们世界的背景。

项目成果

A Full-System VM-HDL Co-Simulation Framework for Servers with PCIe-Connected FPGAs

适用于具有 PCIe 连接 FPGA 的服务器的全系统 VM-HDL 联合仿真框架

• DOI: 10.1145/3174243.3174269
• 发表时间: 2018
• 期刊: FPGA '18 Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
• 作者: Cho, Shenghsun;Patel, Mrunal;Chen, Han;Ferdman, Michael;Milder, Peter
• 通讯作者: Milder, Peter