How to harness the benefits of multisensory-motor integration: Attention and Preparation

听到蚊子，看到蚊子，打蚊子。从保护自己免受昆虫侵害，到拿起手机，再到从水瓶里喝水，自愿的目标导向行动构成了我们日常互动的基础。拟议的对多感觉-运动整合原则的系统探索将改善人类如何执行这些对日常生活、教育、就业和娱乐活动至关重要的自愿行动。能够支持日常任务的智能技术正在变得越来越强大，以使生活更有效率。然而，这类技术仍然需要人类的监督；我们如何整合、解释和回应多感官刺激，直接影响我们在完成日常任务方面的相对成功。我的研究项目试图更好地理解和解释人类在执行自愿行动时如何整合我们的所见所闻。虽然新技术的设计者有帮助人类更有效地执行任务的最佳意图，但有许多方法可以让感官输入的增加也会导致反应更长、更不准确。直到最近，我们所知道的关于多感官整合的大部分知识都是基于简单的反应，比如按下按钮。我们知道，个人和任务的特点会影响我们互动和执行志愿行动的方式。我们根据我们打算做什么、认为我们做了什么以及可用的感官反馈，不断地更新和调整我们的行动。本研究将帮助我们了解听觉刺激如何、何时以及为什么影响运动成绩。我将进行一系列实验，共同评估任务指令如何与听觉刺激相互作用，以及节奏如何影响或稳定运动规划过程。将通过使用凝视跟踪来测量人们看向哪里，以及使用三维运动分析来测量他们如何在空间中移动肢体，从而对如何进行动作进行详细分析。这一基本研究路线的结果将为一系列可能的应用提供信息，包括设计新的人机界面、疗法和教育方法。例如，通过了解节奏如何影响运动，从业者将更好地预测节奏将在何时为谁带来改善的肢体控制。了解多感官刺激如何以及为什么可以促进自愿行动，也将为提高任务学习的训练技术和设计原则提供信息；这对驾驶等活动至关重要，因为在这些活动中，高效和有效的运动对安全至关重要。

Hear mosquito, see mosquito, hit mosquito. From protecting ourselves from insects, to picking up our phone, to drinking from our water bottle, voluntary goal-directed actions form the basis for our everyday interactions. The proposed systematic exploration of the principles for multisensory-motor integration will improve how humans perform these voluntary actions that are essential for activities of daily living, education, employment and recreation. Smart technologies able to support everyday tasks are becoming increasingly enhanced to make life more efficient. However, such technologies still require human oversight; how we integrate, interpret and respond to multisensory stimuli directly impacts our relative success in accomplishing daily tasks. My research program seeks to better understand and explain how humans integrate what we see, feel and hear when performing voluntary actions. While designers of new technology have the best intentions to help humans perform more efficiently, there are numerous ways in which the increasing sensory input may also lead to longer and less accurate responses. Until recently, much of what we knew about multisensory integration was based on simple responses, such as pressing a button. We know that individual and task characteristics influence how we interact and perform voluntary actions. We continuously update and adapt our actions based on what we intended to do, thought we did, and the available sensory feedback. The present research will help us understand how, when and why auditory stimuli impacts movement performance. I will conduct a series of experiments that together assess how task instructions interact with auditory stimuli and how rhythm may influence or stabilize movement planning processes. Detailed analyses of how movements are performed will be done by measuring where people look using gaze tracking and how they move their limbs through space using three-dimensional motion analysis. The results of this fundamental line of research will inform a range of possible applications, including the design of new human computer interfaces, therapies and educational approaches. For example, by understanding how rhythm influences movement, practitioners will be better informed to predict when and for whom rhythm will lead to improved limb control. Understanding how and why multisensory stimuli can facilitate voluntary actions will also inform training techniques and design principles that will enhance task learning; crucial for activities, such as driving, where efficient and effective movement is essential for safety.