US-German Collaboration: Integration of Bottom-Up and Top-Down Signals in Visual Recognition

视觉认知是由携带感官信息的“自下而上”（前馈）过程和在目标、任务、情绪和存储信息的背景下调节输入的“自上而下”（反馈）过程的相互作用协调的。在解剖学层面上，大脑皮层内的每个区域都受到前馈信号和反馈信号的高度神经支配。在美国国家科学基金会的资助下，儿童医院公司（波士顿，马萨诸塞州）的Gabriel Kreiman博士与Freiburg大学医院（Freiburg，德国）的Andreas Schulze-Bonhage博士合作，通过将计算模型和机器学习技术与人类颞叶的高分辨率神经生理记录相结合，研究自下而上和自上而下神经信号的动态集成。研究人员早就认识到，自上而下和自下而上的信号在视觉识别中起着关键作用，然而，这些信号之间的相对贡献和相互作用尚不清楚。该研究项目专注于认知的一个特定方面，即我们视觉识别模式的能力，这是大多数日常任务的核心。即使是目前最好的机器计算模型也只能提供高级视觉皮层复杂神经生理反应的粗略近似。毫不奇怪，一个三岁的孩子在识别任务上的表现可以超过复杂的计算算法，比如在复杂的环境中导航，或者在混乱的场景中识别物体。该研究项目侧重于三个日益复杂的任务，这些任务越来越依赖于自上而下的影响。第一个研究目标涉及在杂乱的视觉刺激中识别物体时自上而下的影响。第二个目的是研究人类颞叶的神经生理反应是否能够支持部分物体信息的识别。这个问题是通过研究客体补全现象来解决的。第三个目标是在复杂的现实识别场景中结合视觉刺激杂波和遮挡。为此，研究人员正在研究癫痫患者在玩一款定制的视频游戏时，注意力和任务相关目标对神经生理活动的影响。这些神经生理学数据利用难得的机会，将高分辨率神经生理学、计算模型和复杂的行为任务结合起来，开展在非人类动物身上很难进行的研究。通过进一步了解感知信息转化为认知，研究人员正在为两个更广泛的目标做出贡献：一个目标是通过开发大脑和机器之间的接口来帮助缓解认知障碍所涉及的挑战性条件，另一个目标是应用有关神经元回路的知识来开发自动从感官数据中提取认知信息的计算算法。建立一个快速、健壮、可靠的人工视觉系统将在许多科学和工程领域产生深远的影响，包括模式识别、监视和安全、自动导航和临床图像分析。这些科学和工程上的进步可以反过来转化为工业伙伴关系感兴趣的重要现实应用。理解视觉系统依赖于许多技能，从计算机科学到物理学、神经科学和心理学。这些研究工作还得到旨在培训跨学科科学家的教育和外联倡议的补充。培训是培养多学科的学生，他们可以建立在他们的基本科学技能，并将这些知识应用到具有挑战性的临床和工程问题。该项目由认知神经科学项目、社会行为与经济学部、计算神经科学合作研究和国际科学与工程办公室共同资助。德国教育和研究部（BMBF）正在资助一个伙伴项目。

Visual cognition is orchestrated by the interaction of 'bottom-up' (feed-forward) processes that carry sensory information and 'top-down' (feed-back) processes that modulate the incoming input in the context of goals, tasks, emotions and stored information. At the anatomical level, each area within the cerebral cortex is heavily innervated by both feed-forward signals and feed-back signals. With funding from the National Science Foundation, Gabriel Kreiman, Ph.D. of Children's Hospital Corporation (Boston, Massachusetts) in collaboration with Andreas Schulze-Bonhage, Ph.D., of the Freiburg University Hospital (Freiburg, Germany), is investigating the dynamical integration of bottom-up and top-down neural signals, by combining computational models and machine learning techniques for data analysis with high-resolution neurophysiological recordings from the human temporal lobe. Researchers have long recognized that top-down and bottom-up signals play a key role in visual recognition, however, the relative contribution and interactions between these signals remain unclear. The research project is focused on a particular aspect of cognition, namely our ability to visually recognize patterns, which is central to most everyday tasks. Even the best machine computational models available today only provide a coarse approximation to the complex neurophysiological responses found in higher visual cortex. Not surprisingly, a three-year-old can outperform sophisticated computational algorithms in recognition tasks, such as navigation in complex environments or recognizing objects in cluttered scenes. The research project focuses on three progressively more complex tasks that rely increasingly on top-down influences. The first research aim involves top-down influences during recognition of objects in a cluttered visual stimulus. The second aim examines whether neurophysiological responses in the human temporal lobe can support recognition from partial object information. This question is being approached through studying the phenomenon of object completion. The third aim combines visual stimulus clutter and occlusion in a complex realistic recognition scenario. For this aim, the researchers are examining the influences of attention and task-related goals on neurophysiological activity while epilepsy patients play a custom-designed video game. These neurophysiological data take advantage of the rare opportunity to combine high-resolution neurophysiology, computational models, and behaviorally complex tasks to carry out research that would be difficult with non-human animals. By furthering the understanding of the transformation of perceptual information into cognition, the researchers are contributing to two broader goals: The goal to help alleviate the challenging conditions involved in cognitive disorders through the development of interfaces between brains and machines, and the goal to apply knowledge about neuronal circuits to develop computational algorithms that automatically extract cognitive information from sensory data. Building a fast, robust, and reliable artificial vision system would have profound repercussions in many areas of science and engineering, including pattern recognition, surveillance and security, automatic navigation and clinical image analysis. These scientific and engineering advances could in turn translate into important real-world applications of interest for industrial partnerships. Understanding the visual system relies on many skills ranging from computer science to physics, neuroscience, and psychology. The research efforts are complemented by educational and outreach initiatives aimed at training interdisciplinary scientists. The training is producing multidisciplinary students who can build on their fundamental scientific skills and apply this knowledge to challenging clinical and engineering problems. This project is jointly funded by the Cognitive Neuroscience Program, the Social Behavioral and Economics Division, Collaborative Research in Computational Neuroscience, and the Office of International Science and Engineering. A companion project is being funded by the German Ministry of Education and Research (BMBF).