ITR/SY: A Neuromorphic Vision System for Every-Citizen Interfaces

ITR/SY：适用于每个公民界面的神经形态视觉系统

• 批准号: 0112991
• 负责人: Christof Koch
• 金额: $ 45万
• 依托单位: California Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2001
• 资助国家: 美国
• 起止时间: 2001-08-15 至 2005-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0112991&HistoricalAwards=false
• 关键词: ITR; SY; Neuromorphic; Vision; System

This project aims to extend an existing simple saliency-based visual attention system to animated color video sequences so as to enable it to cue the object recognition module towards interesting locations in live video streams, and simultaneously to extend an existing model for object recognition to on-line adaptability through top-down signals and task- and object-dependent learning of features. The PIs will then integrate these attention and recognition models, by developing feedforward and feedback interactions between localization of regions of interest and object recognition in those regions. This will require substantial elaboration of both models, as well as specific work on their integration. The result will be a complete model of object localization and recognition in primates, with direct applicability to computer vision challenges. The PIs will next implement and deploy the combined model on a cluster of CPUs linked by very fast interconnect (just installed at USC) to allow for real-time processing, and will demonstrate its utility in a prototype video-conferencing application in which the on-line adaptive attentional component of the integrated system will quickly locate regions in the monitored environment where something interesting is happening (e.g., a user raising her hand in a conference room). The recognition part of the system will then be trained and refined on-line to recognize relatively simple hand signs (e.g., a finger pointing up, meaning that the user wishes to become the center of interest in a video-conference). This work will demonstrate two points: that a biologically-inspired approach to traditionally hard computer vision problems can yield unusually robust and versatile vision systems (which work with color video streams and quickly adapt to various environmental conditions, users, and tasks); and that computational neuroscience models of vision can be extended to yield real, useful and widely applicable computer vision systems, and are not restricted to testing neuroscience hypotheses under simple laboratory stimuli.

该项目旨在将现有的简单的基于显着性的视觉注意系统扩展到动画彩色视频序列，以便使其能够提示对象识别模块在实时视频流中的有趣位置，同时将现有的对象识别模型扩展到在线自适应性，通过自上而下的信号和任务和对象相关的特征学习。 然后，PI将通过在感兴趣区域的定位和这些区域中的对象识别之间开发前馈和反馈交互来集成这些注意力和识别模型。 这将需要对这两种模式进行实质性的阐述，并就其一体化开展具体工作。 其结果将是一个完整的灵长类动物目标定位和识别模型，直接适用于计算机视觉挑战。 PI接下来将在通过非常快速的互连（刚刚安装在USC）连接的CPU集群上实现和部署组合模型，以允许实时处理，并将在原型视频会议应用中展示其实用性，其中集成系统的在线自适应注意力组件将快速定位监控环境中发生有趣事件的区域（例如， 用户在会议室中举手）。 然后，系统的识别部分将被在线训练和改进，以识别相对简单的手势（例如，手指向上指，意味着用户希望成为视频会议中的兴趣中心）。 这项工作将证明两点：生物启发的方法来解决传统的计算机视觉难题，可以产生异常强大和通用的视觉系统（可处理彩色视频流，并快速适应各种环境条件、用户和任务）;并且视觉的计算神经科学模型可以被扩展以产生真实的、有用的和广泛适用的计算机视觉系统，并且不限于在简单的实验室刺激下测试神经科学假设。