Towards Understanding How Active Observers Solve Visuospatial Tasks

这个建议的重点是视觉注意力和主动观察，首先，我们作为人类如何解决复杂的三维空间任务，其次，我们可以创建人工智能来做同样的事情吗？例如，我们正在考虑的任务是如何确定两个对象是否相同（3D相同-不同）？另一个可能是如何在一个大的杂乱空间中找到一个特定的对象（3D视觉搜索）。当然，有时颜色或形状会立即揭示答案。但情况并非总是如此。 有时你的观点会产生很大的不同（比如在著名的错觉或雕塑中）。这种能力每天都在使用，是现实世界机器人助手的核心能力。假设在装配任务期间，您被要求查找在附近箱子中找到的特定零件或工具（例如，家具组装）。儿童的建筑玩具需要一个人在构建一个块配置时多次执行此操作，无论是从计划复制，模仿现有的还是从一个人的想象中构建。解决简单的问题并不那么有趣;人类也可以解决困难的问题。这个问题没有得到很好的研究。有许多实验测试我们在看屏幕上的东西时的视觉能力。然而，人类并没有进化到对坐在椅子上的屏幕上的视觉世界的静态剪切进行分类;他们是他们世界的积极观察者。在这方面取得进展，无论是在实验上还是在计算上，都是这项提议的关键目标。很容易得出结论，注意力必须参与其中;但如何参与？因此，我的整个计划有3个主题：A）视觉注意，B）它在功能视觉中的作用，以及3）它在智能代理中的体现。 无论是生物实现还是人工实现，智能体都必须知道其感知引导行为的原因、方式、时间、地点和内容。决定你为什么看（这样你就知道该怎么处理你所看到的，即，任务指导），看什么（为你的任务选择目标，即，任务相关候选者）、何时查看（我应该在什么时候查看任务相关目标，即，注意动作的序列），从哪里看（以获得正确的视点，即，优化3D成像几何形状），以及如何观看（以适应感觉参数，即，优化成像灵敏度）至少与所见图像的分析同样重要。我们正在努力理解这些不同的功能，开发一种理论来解释它们如何详细工作，并使用该理论开发可以在自然3D世界中成为主动助手的智能代理。 这项工作既有科学影响，也有实际影响。在科学方面，我们希望开发算法，可以将复杂的视觉任务如何解决归纳为我们将要处理的1或2个任务。在实践方面，科学将使我们能够开发有用的机器人助手，如工业装配或家庭护理助手。

This proposal focuses on visual attention and active observation asking first how do we as humans solve complex three-dimensional spatial tasks and second, can we create artificial agents to do the same? The kind of task we are considering, for example, is how does one determine if two objects are the same (3D same-different)? Another might be how to find a particular object in a large cluttered space (3D visual search). Naturally, sometimes colour, or shape reveals the answer immediately. But this is not always the case.  Sometimes your viewpoint makes all the difference (such as in well-known illusions or sculptures). This ability is used daily and is a core competency for real-world robotic assistants. Suppose you are asked to find a specific part or tool during an assembly task, found in a nearby bin (e.g., assembly of furniture). Children's building toys require one to perform this many times while constructing a block configuration, either copying from a plan, mimicking an existing one or building from one's imagination. Solving the easy cases are not so interesting; humans can do the hard ones too. This problem is not well studied. There are many experiments that test our visual ability when looking at something on a screen. However, humans did not evolve to categorize still cut-outs of their visual world on a screen, seated in a chair; they are active observers of their world. Making progress on this, both experimentally and computationally, is the key objective for this proposal. It is easy to conclude that attention must be involved; but how? Thus, my overall program has 3 themes: A) visual attention, B) its role in functional vision, and 3) its embodiment in intelligent agents. Regardless of biological or artificial realization, an intelligent agent must know the why, how, when, where, and what of its perceptually-guided behaviour. Deciding why you look (so you know what to do with what you see, i.e., task guidance), what to look at (to chose targets for your task, i.e., task-relevant candidates), when to look (at what time should I look at a task relevant target, i.e., sequences of attentive actions), where to look from (to get the right viewpoint, i.e., optimizing 3D imaging geometry), and how to look (to adapt sensory parameters, i.e., optimizing imaging sensitivity), is at least as important as the analysis of the seen image. We are working towards understanding these various capabilities, developing a theory to explain how they work in detail, and using that theory to develop intelligent agents that can be active assistants in the natural 3D world. There are both scientific and practical impacts of this work. On the science side we hope to develop algorithms that might codify how complex visual tasks might be solved that would generalize to more than the 1 or 2 tasks we will work on. On the practical side, the science will enable us to develop useful robotic assistants as industrial assembly or home caregiver assistants.