Collaborative Research: Visual Cortex on Silicon

合作研究：硅上视觉皮层

• 批准号: 1317470
• 负责人: Subhasish Mitra
• 金额: $ 60万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-10-01 至 2019-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1317470&HistoricalAwards=false
• 关键词: Collaborative; Research; Visual; Cortex; Silicon

The human vision system understands and interprets complex scenes for a wide range of visual tasks in real-time while consuming less than 20 Watts of power. This Expeditions-in-Computing project explores holistic design of machine vision systems that have the potential to approach and eventually exceed the capabilities of human vision systems. This will enable the next generation of machine vision systems to not only record images but also understand visual content. Such smart machine vision systems will have a multi-faceted impact on society, including visual aids for visually impaired persons, driver assistance for reducing automotive accidents, and augmented reality for enhanced shopping, travel, and safety. The transformative nature of the research will inspire and train a new generation of students in inter-disciplinary work that spans neuroscience, computing and engineering discipline.While several machine vision systems today can each successfully perform one or a few human tasks ? such as detecting human faces in point-and-shoot cameras ? they are still limited in their ability to perform a wide range of visual tasks, to operate in complex, cluttered environments, and to provide reasoning for their decisions. In contrast, the mammalian visual cortex excels in a broad variety of goal-oriented cognitive tasks, and is at least three orders of magnitude more energy efficient than customized state-of-the-art machine vision systems. The proposed research envisions a holistic design of a machine vision system that will approach the cognitive abilities of the human cortex, by developing a comprehensive solution consisting of vision algorithms, hardware design, human-machine interfaces, and information storage. The project aims to understand the fundamental mechanisms used in the visual cortex to enable the design of new vision algorithms and hardware fabrics that can improve power, speed, flexibility, and recognition accuracies relative to existing machine vision systems. Towards this goal, the project proposes an ambitious inter-disciplinary research agenda that will (i) understand goal-directed visual attention mechanisms in the brain to design task-driven vision algorithms; (ii) develop vision theory and algorithms that scale in performance with increasing complexity of a scene; (iii) integrate complementary approaches in biological and machine vision techniques; (iv) develop a new-genre of computing architectures inspired by advances in both the understanding of the visual cortex and the emergence of electronic devices; and (v) design human-computer interfaces that will effectively assist end-users while preserving privacy and maximizing utility. These advances will allow us to replace current-day cameras with cognitive visual systems that more intelligently analyze and understand complex scenes, and dynamically interact with users.Machine vision systems that understand and interact with their environment in ways similar to humans will enable new transformative applications. The project will develop experimental platforms to: (1) assist visually impaired people; (2) enhance driver attention; and (3) augment reality to provide enhanced experience for retail shopping or a vacation visit, and enhanced safety for critical public infrastructure. This project will result in education and research artifacts that will be disseminated widely through a web portal and via online lecture delivery. The resulting artifacts and prototypes will enhance successful ongoing outreach programs to under-represented minorities and the general public, such as museum exhibits, science fairs, and a summer camp aimed at K-12 students. It will also spur similar new outreach efforts at other partner locations. The project will help identify and develop course material and projects directed at instilling interest in computing fields for students in four-year colleges. Partnerships with two Hispanic serving institutes, industry, national labs and international projects are also planned.

人类视觉系统可以实时理解和解释各种视觉任务的复杂场景，同时消耗不到20瓦的功率。该探险项目探索了机器视觉系统的整体设计，这些设计有可能接近并最终超过人类视觉系统的能力。这将使下一代机器视觉系统不仅可以记录图像，还可以理解视觉内容。这种智能机器视觉系统将对社会产生多方面的影响，包括视觉障碍者的视觉辅助工具，减少汽车事故的驾驶员帮助以及增强现实，以增强购物，旅行和安全性。研究的变革性质将激发和培训新一代学生的跨学科工作，这些工作跨越神经科学，计算和工程学科。当今有几个机器视觉系统能否成功执行一项或几项人为任务？例如检测到人头型相机中的人脸吗？他们仍然受到执行各种视觉任务，在复杂，混乱的环境中运作以及为决策提供推理的能力。 相比之下，哺乳动物视觉皮层在各种面向目标的认知任务中出色，并且至少比定制的最先进的机器视觉系统要高至少三个数量级。拟议的研究设想了机器视觉系统的整体设计，该设计将通过开发由视觉算法，硬件设计，人机接口和信息存储组成的全面解决方案来接近人类皮层的认知能力。该项目旨在了解视觉皮层中使用的基本机制，以设计新的视觉算法和硬件面料，这些算法和硬件面料可以提高相对于现有机器视觉系统的功率，速度，灵活性和识别精度。为了实现这一目标，该项目提出了一个雄心勃勃的跨学科研究议程，该议程将（i）了解大脑中目标指导的视觉注意机制来设计任务驱动的视觉算法； （ii）开发视力理论和算法，以增加场景的复杂性，从而扩展性能； （iii）整合生物和机器视觉技术中的互补方法； （iv）开发了一种新的计算体系结构，灵感来自对视觉皮层和电子设备的出现的进步的启发； （v）设计人类计算机界面，这些界面将有效地协助最终用户，同时保留隐私和最大化效用。这些进步将使我们能够用认知视觉系统替换当前的摄像机，从而更智能地分析和理解复杂的场景，并与用户动态互动。以类似于人类的方式理解和与环境交互的机动视觉系统将启用新的变革性应用程序。该项目将开发实验平台至：（1）协助视力受损的人； （2）增强驾驶员的注意； （3）增强现实，为零售购物或度假访问提供增强的体验，并提高关键公共基础设施的安全性。该项目将导致教育和研究文物通过网络门户和在线演讲进行广泛传播。由此产生的工件和原型将增强对代表性不足的少数民族和公众的成功宣传计划，例如博物馆展览，科学博览会和针对K-12学生的夏令营。它还将刺激其他合作伙伴地点的类似新的外展工作。该项目将有助于识别和开发旨在灌输对四年制大学学生的计算领域兴趣的课程材料和项目。还计划了与两个西班牙裔服务机构，行业，国家实验室和国际项目的合作伙伴关系。