Collaborative Research: Multimodal State Estimation through Neural Coherence in the Parieto-Frontal Network

为了将我们身体的某些部位与周围的其他物体区分开来，大脑需要通过将来自皮肤、肌肉和关节的信息与来自眼睛的信息相结合，来建立我们身体的内部图像。该项目旨在通过使用多地点大脑记录和虚拟现实技术来描述我们如何建立自我意识。研究结果将影响消费者、医疗保健、军事和工业环境的国家需求，通过推进基础工程和神经科学知识来创造下一代脑机接口，这些接口被设想为支持人工和自然感官信息的整合。优化这些系统主要取决于理解自然感觉信号如何在大脑区域内和区域之间相互作用，这是该项目直接解决的知识差距。与该项目相关的教育目标旨在促进对工程和神经科学的发现和理解，同时也促进超出这些学科常规范围的教学、培训和学习。实现这些目标的途径是：1)通过开发动手教学模块，吸引在STEM领域代表性不足的高中生；2)通过在亚利桑那州立大学实习，促进跨学科本科生研究机会；3)通过接触从事神经科学研究的伦理、社会和政策影响的组织，指导学生了解科学研究的更广泛含义；4)通过公开讲座和展览让公众参与科学话语，从而促进工作的广泛传播，增进科学和技术的认识。对大多数生物体来说，通过整合可用的感觉线索（多模态估计）来估计身体的状态是一项重要的综合功能。尽管在行为水平上对上肢的状态估计了解很多，但在皮层区域，特别是在网络水平上，潜在的神经机制仍然知之甚少。这是由于以下几个因素：1)被认为在肢体状态估计中起作用的皮层区域在其感觉输入的相对强度方面是异质性的，并且根据环境表现出多感觉增强和抑制；2)技术限制意味着这些领域之间的功能相互作用难以表征；3)视觉和躯体线索敏感性与多感觉统合计算理论之间的关系尚未得到充分探讨；4)多模态区域被认为有助于感知和基于行动的身体表征，但这些表征如何在神经和行为水平上相互作用尚不清楚。因此，目前尚不清楚上肢状态的连贯多模态估计是如何构建和维持的。本文提出的一系列研究通过量化猴子额顶叶区域内和之间的神经尖峰、局部场电位和神经一致性的变化来解决这些问题，这些变化涉及使用虚拟到达任务来改变视觉线索的可靠性和语义信息的状态估计。

To distinguish parts of our body from other objects around us, the brain needs to build an internal image of our body by merging information from the skin, muscles and joints with information from our eyes. This project is aimed at characterizing how we build our sense of self, by using multisite brain recordings and virtual reality technologies. The results will impact national needs in the consumer, healthcare, military, and industrial settings by advancing the fundamental engineering and neuroscience knowledge necessary to create the next generation of brain-machine interfaces, which are envisioned to support the integration of artificial and natural sensory information. Optimizing these systems depends critically on understanding how natural sensory signals interact within and among brain areas, a knowledge gap that directly addressed by this project. The educational goals associated with the project are designed to advance discovery and understanding of engineering and neuroscience, while also promoting teaching, training, and learning beyond the regular bounds of these disciplines. These goals are achieved by: 1) Engaging high school students underrepresented in STEM fields through the development of hands-on instructional modules, 2) Promoting interdisciplinary undergraduate research opportunities via internships at Arizona State University, 3) Mentoring students in the broader implications of scientific research through exposure to organizations engaged in the ethical, societal, and policy implications of neuroscience research, and 4) Engaging the public in scientific discourse through public lectures and exhibits, and thereby promoting broad dissemination of the work to enhance scientific and technological understanding.Estimating the state of the body through the integration of available sensory cues (multimodal state estimation) is a critical integrative function for most organisms. Although much is known about state estimation for the upper limb at the behavioral level, the underlying neural mechanisms remain poorly understood in cortical areas, particularly at the network level. This is due to several factors: 1) the cortical areas believed to play a role in limb state estimation are heterogenous with regard to the relative strength of their sensory inputs and display both multisensory enhancement and suppression depending on context; 2) technical limitations mean functional interactions among these areas have been challenging to characterize; 3) the relation between sensitivity to visual and somatic cues and prevailing computational theories of multisensory integration have been incompletely explored; 4) multimodal areas are thought to contribute to both perceptual and action-based body representations but how these representations interact at the neural and behavioral levels is not well understood. As a result, it is unclear how a coherent multimodal estimate of the state of the upper limb is constructed and maintained. The proposed series of studies address these issues by quantifying changes in neural spiking, local field potentials, and neural coherence within and among fronto-parietal areas of the monkey implicated in state estimation using virtual reaching tasks that alter the reliability and semantic information of visual cues.