Attention and Selection Mechanisms in Neural Networks and Computer Vision

拟议的研究是由神经信息处理系统，包括视觉问题的顺序解决方案的注意，选择，门控和路由信息的机制。循环的深度学习模型形成了展示所提出的工作语料库的价值和实用性的主要结构。虽然深度学习模型在广泛的问题上取得了相当大的成功，但仍然需要更好地描述其行为，并在产生解决方案的方向上进行创新，这些解决方案在实现其目标方面更加简约和有效。为此，本提案概括了为实现这些目标而开展的一整套理论和实验工作。这是通过两个不同但互补的支柱作为基础来实现的：首先，我们建议详细研究通过神经层次控制信息流的循环门控机制。这包括添加剂。乘法和其他门控机制，它们与增益控制和归一化策略的关系，以及与常见前馈网络中出现的规范机制的联系。我的研究小组已经显示出这种控制机制对深度神经网络能力的巨大潜力。其次，我们解决需要考虑的问题，包括顺序或时间识别相关模式或显着的区域的解决方案。视觉本质上是连续的，最明显的例子就是人类的头部和眼睛的运动。同样重要的是大脑内部处理的动态性质，包括注意力的作用。到目前为止，计算机视觉中的许多问题都是以一种在一次通过中产生对图像的像素分配的方式操作的。第二个支柱涉及仔细检查空间中的注意选择，以及信息处理的循环机制，以促进视觉处理。这项调查的产品将包括顺序模型的凝视模式，解决视觉问题的顺序方式，焦点处理和经常性门控和比较人类和机器视觉解决方案的空间采样之间的关系。这项工作的每一个方面都从确立重要的基本原则作为基础开始，同时审查在经常性网络中的实际适用性，并与关注机制建立联系。总的来说，拟议的研究将从架构的角度为深度学习提供基础性贡献，并在增强我们对神经网络信息处理能力的理解的层面上。此外，该研究将现状扩展到更广泛的机器视觉问题的顺序或时间解决方案，作为解决问题的替代范式，并以适合主动视觉系统的形式。

The proposed research is comprised of mechanisms for attention, selection, gating and routing of information in neural information processing systems including sequential solutions to visual problems. Recurrent deep learning models form the primary construct for demonstrating the value and utility of the proposed corpus of work. While deep learning models have shown considerable success on a wide range of problems, there remains a need for better characterization of their behaviour, and innovation in directions that produce solutions that are both more parsimonious and effective in addressing their objectives. To this end, this proposal encapsulates a comprehensive body of theoretical and experimental work to meet these objectives. This is achieved through two distinct but complementary pillars as a foundation: First, we propose to examine in detail recurrent gating mechanisms that control flow of information through neural hierarchies. This includes additive. multiplicative and other gating mechanisms, their relation to gain control and normalization strategies, and connections to canonical mechanisms that appear in common feed-forward networks. My research group has shown significant potential for such control mechanisms for the capabilities of deep neural networks. Second, we address the need to consider solutions to problems that include sequential or temporal identification of relevant patterns or salient regions. Vision is inherently sequential with the most palpable example of this in humans involving head and eye movements. Equally important is the dynamic nature of processing within the brain including the role of attention. To date, many problems in computer vision operate in a manner that produces pixel-wise assignments to an image in one pass. This second pillar involves careful examination of attentive selection in space, and recurrent mechanisms for information processing to facilitate processing for vision. Products of this investigations will include sequential models of gaze patterns, solution to vision problems in a sequential manner, relationships between focal processing and recurrent gating and comparison of human an machine vision solutions to spatial sampling. Each facet of this work begins with establishing basic principles of importance as a foundation while examining practical applicability in recurrent networks and also drawing connections to attention mechanisms. As a whole, the proposed research stands to provide fundamental contributions to deep learning from an architectural perspective and at the level of augmenting our understanding of information processing capabilities of neural networks. Moreover, the research extends the status quo into a broader set of sequential or temporal solutions to machine vision problems, as an alternative paradigm for problem solving and in a form amenable to active vision systems.