VALUE: Vision, Action, and Language Unified by Embodiment

这个项目的主要目的是模拟解决以下问题的过程：如何根据智能体的知识和对世界的表示，从几个对象中选择一个对象，掌握对象并以适当的方式使用它。这个平凡的活动实际上需要同时解决不同层次的几个深层问题。智能体的视觉系统必须表示潜在的目标对象，目标必须根据任务指令或智能体对所表示对象的功能的了解来选择，并且手（在这种情况下）必须移动到目标并形成形状，以便以适合其使用的方式抓住它。我们建议开发一个机器人仿真模型，灵感来自于最近的具身认知理论，其中视觉，动作和语义系统在连接主义架构中以动态和相互交互的方式联系在一起。人类的实验工作将限制系统的时间和动态特性，以努力发展一个心理学上可信的行动具体化选择模型。由于许多导致动作和视觉之间整合的认知机制（如物体组装任务）尚不完全清楚，本项目中的新实证研究也将提高我们对这些具体认知动力学的认识。新的实验和具身认知模型的使用也将用于进一步我们对语言和认知整合的理解，例如，通过对对象表示中语言和行动知识的动态提供进一步的预测和见解。这是一个跨学科项目，涉及认知心理学、运动控制和计算/机器人建模的专业知识和方法。该项目的跨学科性质以及认知代理的设计和实验使该项目与认知系统预见计划高度相关

The primary aim of this project is to develop a simulation of the processes involved in solving the following problem: how to select, based on the agent's knowledge and representations of the world, one object from several, grasp the object and use it in an appropriate manner. This mundane activity in fact requires the simultaneous solution of several deep problems at various levels. The agent's visual system must represent potential target objects, the target must be selected based on task instructions or the agent's knowledge of the functions of the represented objects, and the hand (in this case) must be moved to the target and shaped so as to grip it in a manner appropriate for its use. We propose to develop a robotic simulation model inspired by recent theories of embodied cognition, in which the vision, action and semantic systems are linked together, in a dynamic and mutually interactive manner, within a connectionist architecture. Human experimental work will constrain the temporal and dynamic properties of the system in an effort to develop a psychologically plausible model of embodied selection for action. As much of the cognitive mechanisms leading to the integration between action and vision for actions such as object assembly tasks are not fully known, new empirical studies in this project will also improve our insight of these embodied cognitive dynamics. New experiments and the use of the embodied cognitive model will also be used to further our understanding of language and cognition integration e.g. by providing further predictions and insights on the dynamics of language and action knowledge in object representation.This is an interdisciplinary project which involves expertise and methodologies from cognitive psychology, motor control, and computational/robotics modelling. The interdisciplinary nature of the project and the design and experimentation of cognitive agents make the project highly relevant to the Cognitive Systems Foresight programme