VEC: Medium: Large-Scale Visual Recognition: From Cloud Data Centers to Wearable Devices

VEC：中：大规模视觉识别：从云数据中心到可穿戴设备

• 批准号: 1539011
• 负责人: Thomas Wenisch
• 金额: $ 96万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1539011&HistoricalAwards=false
• 关键词: VEC; Medium; Large; Scale; Visual

Advances in computer hardware and software promise to revolutionize the ways in which society interacts with visual information. However, visual recognition systems are limited by the lack of a practical means to classify the millions of concepts that arise in visual scenes and thus efficiently recognize when a small number of these concepts appear in a given scene. Furthermore, while real-time processing of visual data could significantly expand our perception of our surroundings, state-of-the-art vision systems cannot currently be implemented on wearable devices such as smartphones due to the limited heat dissipation (e.g., no fans or liquid cooling) and power such devices can provide. This research will overcome these challenges by developing artificial intelligence (AI) systems that efficiently manage the resources most crucial for high-performance wearable-based visual recognition, including the wearable device's real-time power consumption and computation. These systems will be empowered to initiate bursts of intense computation that are thermally managed by materials within the wearable device which are engineered to melt during heavy heating and solidify between bursts. Moreover, the AI systems will govern the communication between the device and external (cloud-based) computation resources as well as large-scale visual concept databases housed in data centers, thus providing extreme performance in a wearable form factor. Central concepts of this work will be integrated in undergraduate and graduate coursework, and a demonstration system will be made available to the research community and used in educational modules for high school students.This effort seeks to advance the core capabilities of large-scale visual recognition by co-designing visual models and computing infrastructure. The goal is to enable encyclopedic, real-time visual recognition through seamless integration of visual computing on wearable devices and in the cloud. The PIs envision a wearable visual recognition system that continuously captures live video input while providing intelligent, real-time assistance through automatic or on-demand visual recognition by means of a combination of computation at the device and offloading to the cloud. Such a system is not currently feasible due to a number of fundamental challenges. First, the severe energy and thermal constraints of wearable devices render them incapable of performing the intensive computation necessary for visual recognition. Second, it remains an open question how to support encyclopedic recognition in terms of both visual models and data center infrastructure. In particular, it remains unclear how current visual models, although highly successful at recognizing 1,000 object categories, can scale to millions or more distinct visual concepts. Moreover, such an encyclopedic visual model must be supported through data center infrastructure, but little progress has been made on how to build such infrastructure. This project addresses these fundamental challenges through an interdisciplinary approach integrating computer vision, hardware architecture, VLSI design, and heat transfer. The PIs will investigate three research thrusts. In Thrust 1, the PIs will develop a new type of deep neural networks that allow resource-efficient execution of modules. This new framework provide a unified way to design, learn, and run scalable visual models that can maximize the utility of recognition subject to resource constraints, such as latency, energy, or thermal dissipation of a wearable device. In Thrust 2, the PIs will design and fabricate a visual processing chip capable of computational sprinting (bursts of extreme computation well above steady-state thermal dissipation capabilities), leveraging the new framework developed in Thrust 1. In Thrust 3, the PIs will design datacenter infrastructure that supports large-scale hierarchical indexing of visual concepts for encyclopedic recognition, with a focus on latency, throughput, and energy efficiency. Finally, the PIs will build a demonstration system to evaluate the proposed algorithms, software, and hardware components and to assess the overall performance of an end-to-end system. The project web site (http://mivec.eecs.umich.edu/) will provide access to the results of this research including technical reports, datasets, and source code.

计算机硬件和软件的进步有望彻底改变社会与视觉信息交互的方式。然而，视觉识别系统受到限制，因为缺乏实用的方法来对视觉场景中出现的数百万个概念进行分类，从而有效地识别给定场景中出现的少量概念。此外，虽然视觉数据的实时处理可以显着扩展我们对周围环境的感知，但由于此类设备可提供的散热（例如，没有风扇或液体冷却）和电力有​​限，目前最先进的视觉系统无法在智能手机等可穿戴设备上实现。这项研究将通过开发人工智能（AI）系统来克服这些挑战，该系统可以有效管理对高性能可穿戴视觉识别最关键的资源，包括可穿戴设备的实时功耗和计算。这些系统将能够启动密集计算的爆发，这些计算由可穿戴设备内的材料进行热管理，这些材料被设计为在剧烈加热期间熔化并在爆发之间固化。此外，人工智能系统将管理设备与外部（基于云的）计算资源以及数据中心内的大规模视觉概念数据库之间的通信，从而以可穿戴的形式提供极致的性能。这项工作的核心概念将融入本科生和研究生课程，并向研究界提供演示系统，并用于高中生的教育模块。这项工作旨在通过共同设计视觉模型和计算基础设施来提高大规模视觉识别的核心能力。目标是通过可穿戴设备和云中视觉计算的无缝集成来实现百科全书式的实时视觉识别。 PI 设想了一种可穿戴视觉识别系统，该系统可以连续捕获实时视频输入，同时通过自动或按需视觉识别（通过设备计算和卸载到云端的组合）提供智能实时帮助。由于存在许多根本性挑战，这样的系统目前还不可行。首先，可穿戴设备严格的能量和热限制使其无法执行视觉识别所需的密集计算。其次，如何在视觉模型和数据中心基础设施方面支持百科全书式识别仍然是一个悬而未决的问题。特别是，目前的视觉模型虽然在识别 1000 个对象类别方面非常成功，但如何能够扩展到数百万或更多不同的视觉概念，目前仍不清楚。而且，这样一个百科全书式的视觉模型必须通过数据中心基础设施来支持，但在如何构建这样的基础设施方面却进展甚微。该项目通过集成计算机视觉、硬件架构、VLSI 设计和传热的跨学科方法来解决这些基本挑战。 PI 将调查三个研究重点。在 Thrust 1 中，PI 将开发一种新型深度神经网络，以实现模块的资源高效执行。这个新框架提供了一种统一的方式来设计、学习和运行可扩展的视觉模型，可以最大限度地提高受资源限制（例如可穿戴设备的延迟、能量或散热）的识别效用。在 Thrust 2 中，PI 将利用 Thrust 1 中开发的新框架，设计和制造能够进行计算冲刺（远高于稳态散热能力的极限计算爆发）的视觉处理芯片。在 Thrust 3 中，PI 将设计数据中心基础设施，支持用于百科全书式识别的视觉概念的大规模分层索引，重点关注延迟、吞吐量、 和能源效率。最后，PI 将构建一个演示系统来评估所提出的算法、软件和硬件组件，并评估端到端系统的整体性能。该项目网站 (http://mivec.eecs.umich.edu/) 将提供对这项研究结果的访问，包括技术报告、数据集和源代码。