SBE-UKRI: Integrating vision and action through selection history

大多数现实世界的视觉场景都是复杂而拥挤的，其中有多个物体在争夺注意力和目标导向的行动。例如，在日常生活中，一个人很容易从杂货店陈列的各种水果中拿起一个红苹果。与这种复杂环境的成功交互需要多个机制之间的无缝协调。特别是，帮助我们理解世界的注意力选择机制与构成行动选择基础的机制协同工作，使我们能够产生适应性动作。这种注意-行动协同作用是所有复杂行为的根源。对象选择不仅受知觉显著性（自下而上）和当前目标（自上而下）这两个公认因素的指导，还受选择历史的影响。然而，在现实世界中，选择历史与视觉引导行为之间的联系还没有得到充分的研究。为了克服这一差距，本提案的目标是确定控制注意选择和行动选择的机制之间的相互作用，特别是当最近的选择历史对后续行为产生偏差时。该项目将重点关注行动选择的三个未被充分研究的方面：行动执行的变化、效应器和生物力学成本，以确定它们与感知特征选择历史的丰富研究之间的新关系。为确保取得成功，两个研究团队将展开国际合作，发挥互补和协同作用：布朗大学（US-PI）的宋博士是注意力和运动控制方面的专家，他将在人类身上进行心理物理实验，包括连续跟踪和目标导向动作的力场操纵。为了将经验证据转化为可测试的模型，伯明翰大学（UoB） （UK-PI）的计算建模专家Heinke博士将为机械臂（机器人模型）开发和实施生物学上合理的控制体系结构。这一共同努力将促进我们对注意力和行动驱动机制之间相互依赖的理解，最终解释适应性的、现实世界的选择行为。

Most real-world visual scenes are complex and crowded, where multiple objects compete for attention and goal-directed action. In daily life, for example, a person easily picks up a red apple from a grocery display containing many kinds of fruit. Successful interactions with such complex environments require seamless coordination among multiple mechanisms. In particular, mechanisms of attentional selection that help us make sense of the world work in unity with those that underlie action selection, allowing us to generate adaptive movements. This attention-action synergy is at the root of all complex behaviour. Object selection is guided not only by the well-established factors of perceptual salience (bottom-up) and current goals (top-down), but also selection history. Yet, how selection history links to visually-guided actions has been understudied in real-world scenarios. To overcome this gap, the goal of this proposal is to determine the interplay between mechanisms controlling attentional selection and action selection, particularly when recent selection history biases subsequent behaviour. The proposed project will focus on three understudied aspects of action selection: variation in action execution, effectors, and biomechanical costs, to determine their novel relations with the wealth of research on selection history of perceptual features. To ensure successful outcomes, an international collaboration between the two research teams, who will play complementary and synergistic roles, will be formed: Dr Song at Brown University (US-PI) - an expert in attention and motor control - will carry out psychophysical experiments in humans including continuous tracking and force field manipulation of goal-directed actions. To translate empirical evidence to testable models, Dr Heinke at University of Birmingham (UoB) (UK-PI) - a computational modelling expert - will develop and implement biologically plausible control architectures for a robot arm (robotics models). This joint endeavour will advance our understanding of the interdependence between attention and action-driven mechanisms to eventually explain adaptive, real-world selection behaviour.