Visuospatial modulation of bimanual touch perception in real and virtual environments

为了使用我们的手来操纵物体，神经系统必须能够协调和解释大量不同的感觉信息，比如我们的手的位置，手指的排列和位置，以及每个手指接触的东西。由于两只手的信息必须相对于它们的空间位置进行解释，因此计算变得复杂。例如，当双手握在一起时，左手和右手食指占据相邻的空间，但当手臂分开时，左手和右手食指可能占据非常不同的空间。在这些不同的情况下，我们可以依靠我们的视觉来帮助解释我们的触觉。视觉可以提供有关我们的手在空间中的位置以及我们触摸和操纵哪些物体的信息。尽管视觉对于解释手中的触摸信息具有明显的重要性，但视觉影响发生的具体方式仍然知之甚少。本研究旨在探讨简单视觉线索和复杂视觉线索对双手作业中双手触觉信息的影响。了解双手触摸处理将促进人与智能机器之间有效的非语言物理交流，并将支持高度灵巧的机器人系统、先进的神经假体和感觉运动康复策略的开发。确定视觉如何影响我们在真实和虚拟环境中对身体的感知，对社会使用身临其境和互动技术具有明显的影响。该项目还将促进年轻女性在神经科学和相关STEM领域的参与和教育。这项研究计划使用严格的心理物理学、虚拟现实技术和计算建模来阐明简单和复杂的视觉空间线索对双手触摸感知的影响。将进行行为学实验，以量化简单闪光对触觉感知和学习的影响。神经网络模型将与行为数据相适应，以推断神经系统如何调节视觉触觉和空间互动，以及神经回路如何通过学习来修改。复杂的视觉线索对本体感觉和双手触摸的影响也将通过在虚拟现实中进行心理物理实验来检验。具体地说，视觉本体感觉线索和虚拟物体信息将在虚拟现实设计中被改变，允许研究者测试关于本体感觉和因果推理处理的多感觉整合的假设。总体而言，该研究计划将对视觉信息如何调节在两只手上体验的触觉提示的感知产生新的见解。由M3X(工程)和PAC(社会、行为和经济科学)计划共同资助该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In order to use our hands to manipulate objects, the nervous system must be able to coordinate and interpret a large amount of disparate sensory information about the position of our hands, the arrangement and positioning of the fingers, and what each is contacting. The computations are complicated by the fact that the information from the two hands must be interpreted relative to their spatial position. For example, the left and right index fingers occupy adjacent spaces when the hands are held together but the fingers can occupy vastly different spaces when the arms are spread apart. In these different situations, we can conceivably rely on our sense of vision to help interpret our sense of touch. Vision can provide information about where our hands are in space as well as what objects we touch and manipulate. Despite the obvious importance of vision for interpreting touch information from the hands, the specific ways in which the visual influence occurs remains poorly understood. This research is aimed at investigating the effects of simple and complex visual cues on touch information from the two hands in bimanual tasks. Understanding bimanual touch processing will facilitate effective nonverbal physical communication between people and intelligent machines and it will support the development of highly dexterous robotic systems, advanced neuroprosthetics, and sensorimotor rehabilitation strategies. Establishing how vision influences our perception of our body in real and virtual environments has clear implications for society’s use of immersive and interactive technologies. The project will also promote the participation and education of young women in neuroscience and related STEM fields. This research program uses rigorous psychophysics, virtual reality technologies, and computational modeling to elucidate the effects of simple and complex visuospatial cues on the perception of bimanual touch. Behavioral experiments will be performed to quantify the effects of simple light flashes on tactile perception and learning. Neural network models will be fitted to the behavioral data to make inferences about how the nervous system mediates visuotactile and spatial interactions and how neural circuits may be modified through learning. The effects of complex visual cues on proprioception and bimanual touch will also be examined by performing psychophysics experiments in virtual reality. Specifically, visual proprioception cues and virtual object information will be altered in a virtual reality design, allowing the investigator to test hypotheses regarding multisensory integration for proprioception and causal inference processing. Collectively, the research program will yield novel insight into how visual information modulates the perception of tactile cues experienced over the two hands.Co-funded by the M3X (Engineering) and PAC (Social, Behavioral, and Economic Sciences) ProgramsThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.