Understanding human action as observable cognition using combined motion-tracking, eye-tracking and EEG

通常将人类行为描述为一个连续的过程：首先我们接收外部刺激(知觉)，然后我们选择和处理相关信息(认知)，最后我们产生动作(动作)。虽然方便，但这种简单的概念化未能捕捉到真实神经处理和行为的丰富动态。在这个提案中，我将探讨这种传统思维的局限性之一，并演示如何最好地将行动理解为认知的一个可观察的组成部分，而不是一系列过程中孤立的最后一步。这一重要的新观点表明，直接反映我们思维过程的不仅仅是我们所做的行动，还有我们如何执行这些行动的细微之处。这一大规模的长期目标将在一个分为两个较小规模的研究主题的研究计划中进行调查。每个主题都将由一个研究小组进行调查，研究小组由一名研究生领导，其他研究生和本科生共同贡献。 在第一个主题中，我将通过两个研究问题建立通过神经成像观察到的认知和通过行动观察到的认知之间的相互联系，这两个问题直接建立在我之前的理论和方法专长的基础上：1)目标选择和避障的时间进程是什么？2)对象的奖赏或损失价值如何影响其加工？由此产生的实验将使用全新的方法，将触角跟踪、眼睛跟踪和脑电记录结合在一起。这种整合提供了几个关键的优势：第一，它将为“行动作为可观察的认知”假说提供令人信服的趋同证据。第二，它将扩展研究生的方法工具包。第三，它将允许探索以前不可能实现的相关性。例如，我们现在将有能力在试验的基础上看到特定的神经标记如何与眼睛看的地方和手的移动方式相关。 在第二个主题中，我将测试一个耐人寻味的暗示，即内在的想法可以在运动中检测到，以及这些“身体想法”是否可以与另一个人交流。我将再次使用两个研究问题：1)奖励信息(例如，分配给目标的点数)是否反映在运动中？2)个人的享乐偏好是否反映在运动中？由此产生的实验将使用伸展和眼球跟踪以及两种新方法的组合：人感知视频(其中一个实验中参与者的视频成为第二个实验中参与者的刺激)和二元研究(两个参与者一起执行到达任务，以了解他们的相互运动如何相互影响)。 该研究项目旨在为本科生和研究生提供精英培训。它将在一个尖端的研究实验室进行，采用世界上独一无二的综合方法。最重要的是，这些大胆的新研究方向将在基于协作、激情和融合技术原则的研究氛围中进行研究。这项工作的首要目标是用传统的行为观点直接面对问题，以便开辟令人兴奋和广泛的研究途径。例如，它通过了解检测和交流身体思维的潜在问题来更好地了解自闭症谱系障碍，通过利用新技术从行动中识别人类思维来改善人机界面，并向与运动障碍患者合作的临床医生提供更好的诊断信息。

It is conventional to characterize human behaviour as a serial process: first we receive external stimuli (Perception), then we select and process relevant information (Cognition) and finally we produce movements (Action). While convenient, this simple conceptualization fails to capture the rich dynamics of real neural processing and behaviour. In this proposal, I will explore one of the limitations of this conventional thinking and demonstrate how action is best understood as an observable component of cognition, rather than an isolated, last step in a serial process. This important and new idea suggests that it is not just what actions we make, but also the subtleties of how we perform them that directly reflects our thought processes. This large-scale, long term objective will be investigated in a research program divided into two smaller-scale research themes. Each theme will be investigated by a research team, led by a graduate student with collaborative contributions from other graduate and undergraduate students. In the first theme, I will establish reciprocal links between cognition observed through neuroimaging and cognition observed in action through two research questions that directly build on my previous theoretical and methodological expertise: 1) What is the time course of target selection and obstacle avoidance? and 2) How does the reward or loss value of an object affect its processing? The resulting experiments will use completely novel methods, integrating reach-tracking, eye-tracking and electroencephalography. This integration offers several key advantages: First, it will provide compelling convergent evidence for the "action as observable cognition" hypothesis. Second, it will expand graduate students’ toolkit of methods. Third, it will allow previously impossible correlations to be explored. For example, we will now have the power to see, on a trial by trial basis, how a specific neural marker correlates with where the eye is looking and how the hand is moving. In the second theme I will test the intriguing implication that inner thoughts are detectable in movements and whether these “body thoughts” are communicable to another person. I will again use two research questions: 1) Is reward information (e.g. points assigned to targets) reflected in movement? and 2) Are individual hedonic preferences reflected in movement? The resulting experiments will use a combination of reach- and eye-tracking as well as two novel methods: person perception videos (where videos of participants in one experiment become the stimuli for participants in a second experiment) and dyad studies (where two participants perform a reaching task together to see how their mutual movements affect one another). This research program is designed to provide elite training for both undergraduate and graduate students. It will be conducted in a cutting edge research laboratory with integrated methods that are unique in the world. Most importantly, these bold new research directions will be investigated in a research atmosphere based on the principles of collaboration, passion and convergent techniques. The overarching goal of this work is to directly confront problems with the conventional view of behaviour in order to open up exciting and wide ranging research avenues. For instance, it leads to better understanding of autism spectrum disorders by understanding potential problems detecting and communicating body thoughts, better human computer interfaces by harnessing new technologies to recognize human thought from action and better diagnostic information to clinicians working with patients with motor deficits.