Towards Understanding How Active Observers Solve Visuospatial Tasks

该提案侧重于视觉注意和主动观察，首先问我们作为人类如何解决复杂的三维空间任务，其次，我们能否创造人工智能体来做同样的事情？例如，我们正在考虑的任务是，如何确定两个对象是否相同（3D相同-不同）？另一个可能是如何在一个大而杂乱的空间中找到一个特定的对象（3D视觉搜索）。当然，有时颜色或形状会立即揭示答案。但情况并非总是如此。有时候，你的观点会决定一切（比如在著名的幻觉或雕塑中）。这种能力每天都在使用，是现实世界机器人助手的核心竞争力。假设您被要求在组装任务中找到一个特定的部件或工具，在附近的垃圾箱中找到（例如，家具组装）。儿童建筑玩具需要一个人在构建积木配置时多次执行此操作，要么从计划中复制，模仿现有的一个，要么从一个人的想象中构建。解决简单的案子不是那么有趣；人类也可以做困难的事情。这个问题没有得到很好的研究。当我们看屏幕上的东西时，有很多实验可以测试我们的视觉能力。然而，人类并没有进化到对坐在椅子上的屏幕上的视觉世界进行分类；他们是自己世界的积极观察者。在实验和计算上取得进展是本提案的关键目标。很容易得出这样的结论：注意力必须参与其中；但如何?因此，我的整体计划有3个主题：A)视觉注意，B)它在功能视觉中的作用，以及3)它在智能代理中的体现。无论生物实现还是人工实现，智能体必须知道为什么、如何、何时、何地以及它的感知引导行为是什么。决定你为什么看（这样你就知道如何处理你所看到的，即任务指导），看什么（为你的任务选择目标，即任务相关的候选人），什么时候看（我应该在什么时候看一个任务相关的目标，即，注意的动作序列），从哪里看（获得正确的观点，即，优化3D成像几何），以及如何看（适应感官参数，即，优化成像灵敏度），至少和对所见图像的分析一样重要。我们正在努力理解这些不同的功能，开发一种理论来详细解释它们是如何工作的，并利用该理论开发智能代理，这些智能代理可以在自然的3D世界中成为主动助手。这项工作既有科学影响，也有实际影响。在科学方面，我们希望开发一种算法，可以编纂复杂的视觉任务是如何解决的，这种算法可以推广到我们将要处理的1到2个任务之外。在实践方面，科学将使我们能够开发有用的机器人助手，如工业装配或家庭护理助理。

This proposal focuses on visual attention and active observation asking first how do we as humans solve complex three-dimensional spatial tasks and second, can we create artificial agents to do the same? The kind of task we are considering, for example, is how does one determine if two objects are the same (3D same-different)? Another might be how to find a particular object in a large cluttered space (3D visual search). Naturally, sometimes colour, or shape reveals the answer immediately. But this is not always the case.  Sometimes your viewpoint makes all the difference (such as in well-known illusions or sculptures). This ability is used daily and is a core competency for real-world robotic assistants. Suppose you are asked to find a specific part or tool during an assembly task, found in a nearby bin (e.g., assembly of furniture). Children's building toys require one to perform this many times while constructing a block configuration, either copying from a plan, mimicking an existing one or building from one's imagination. Solving the easy cases are not so interesting; humans can do the hard ones too. This problem is not well studied. There are many experiments that test our visual ability when looking at something on a screen. However, humans did not evolve to categorize still cut-outs of their visual world on a screen, seated in a chair; they are active observers of their world. Making progress on this, both experimentally and computationally, is the key objective for this proposal. It is easy to conclude that attention must be involved; but how? Thus, my overall program has 3 themes: A) visual attention, B) its role in functional vision, and 3) its embodiment in intelligent agents. Regardless of biological or artificial realization, an intelligent agent must know the why, how, when, where, and what of its perceptually-guided behaviour. Deciding why you look (so you know what to do with what you see, i.e., task guidance), what to look at (to chose targets for your task, i.e., task-relevant candidates), when to look (at what time should I look at a task relevant target, i.e., sequences of attentive actions), where to look from (to get the right viewpoint, i.e., optimizing 3D imaging geometry), and how to look (to adapt sensory parameters, i.e., optimizing imaging sensitivity), is at least as important as the analysis of the seen image. We are working towards understanding these various capabilities, developing a theory to explain how they work in detail, and using that theory to develop intelligent agents that can be active assistants in the natural 3D world. There are both scientific and practical impacts of this work. On the science side we hope to develop algorithms that might codify how complex visual tasks might be solved that would generalize to more than the 1 or 2 tasks we will work on. On the practical side, the science will enable us to develop useful robotic assistants as industrial assembly or home caregiver assistants.