Real-Time Multisensory Integration for Time-Varying Signals

时变信号的实时多感官集成

• 批准号: 8584124
• 负责人: Benjamin A Rowland
• 金额: $ 7.65万
• 依托单位: WAKE FOREST UNIVERSITY HEALTH SCIENCES
• 依托单位国家: 美国
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-07-01 至 2015-06-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/8584124
• 关键词: Affect; Animals; Architecture; Basic Science; Behavior; Behavioral; Belief; Biological Assay; Brain; Cerebral cortex; Child; Classification; Clinical; Complex; Cues; Diagnosis; Elderly; Environment; Event; Experimental Designs; Feedback; Future; Human; Joints; Judgment; Laboratories; Light; Modality; Modeling; Monitor; Motor; Nature; Noise; Output; Pattern; Pattern Recognition; Perception; Performance; Peripheral; Population; Process; Property; Quality of life; Recurrence; Relative (related person); Response to stimulus physiology; Sensory; Sensory Process; Series; Shapes; Signal Transduction; Solutions; Source; Specific qualifier value; Speed; Staging; Stimulus; Structure; Task Performances; Testing; Time; To specify; Vision; Weight; auditory stimulus; base; clinically significant; cognitive function; computerized data processing; environmental change; expectation; experience; improved; information processing; insight; interest; multisensory; operation; public health relevance; research study; response; statistics; time use; visual stimulus

DESCRIPTION (provided by applicant): Traditional models of perception have a foundational principle that sensory processing is hierarchical: sensory filters decompose the physical world into primitive features that are recombined through a series of stages into a coherent supramodal representation. These models are typically conceptualized as processing time-invariant (i.e., constant) signals or representations, and are evaluated in experiments whose structure (baseline->stimulus->response) parallels the discrete and staged nature of the presumed underlying processing dynamics. Real environments contain continuous time-varying signals and demand continuous perceptual and behavioral solutions, and while it is assumed that the operations of time-invariant models will trivially generalize to the continuous time domain, there are reasons to suspect that they will not. For example, in a continuous, dynamic environment, fluctuations in input traces may be interpreted as either representing noise contamination or changes in the source signals. How the brain achieves stable perceptual solutions in such circumstances is of critical interest. The present application describes a series of short-term experiments that seek to evaluate this issue in the context of multisensory integration. The merging of information across the senses has been shown to improve perceptual and behavioral judgments and speed the responses to external events, particularly those whose unisensory representations are significantly contaminated or obscured by noise. To realize these benefits, the brain must coordinate activity across senses that have different operational parameters, in particular, different reliabilities that are dependent on the immediate environmental circumstances. To achieve optimal integration, the brain must appropriately "weight" signals derived from different senses according to these reliabilities in their joint consideration. The framework proposed here seeks to understand this phenomenon by evaluating whether subjects use Kalman filter dynamics to achieve optimal performance on a continuous-time multisensory task. Subjects are tasked with tracking dynamic and noise-corrupted patterns of visual and auditory stimuli presented alone or in concert. The ability of subjects to adapt to changes in the signal and the relative reliabilities of the senses are interpreted in the context of the proposed model framework. Subject expectations are shaped by prior experience and forewarning of changes in the signals and signal statistics, and the timing and impact of the adjustments they make in their responses consequent to this information is evaluated. The experimental approach applied in the proposed studies will provide important insights into the principles upon which sensory channels are combined and help to establish the boundary conditions upon which real-time integration is achieved.

描述（由申请人提供）：感知的传统模型具有感觉处理是分层的基本原则：感觉过滤器将物理世界分解为原始特征，这些原始特征通过一系列阶段重新组合成连贯的超模态表示。这些模型通常被概念化为处理时不变（即，恒定）信号或表示，并在其结构（基线->刺激->反应）与假定的潜在处理动态的离散和阶段性质平行的实验中进行评估。真实的环境包含连续的时变信号，并需要连续的感知和行为解决方案，虽然假设时不变模型的操作将平凡地推广到连续时域，但有理由怀疑它们不会。例如，在连续的动态环境中，输入轨迹的波动可以被解释为表示噪声污染或源信号的变化。大脑如何在这种情况下实现稳定的感知解决方案是至关重要的。本申请描述了一系列 短期实验，旨在评估这一问题的背景下，多感官整合。跨感官的信息融合已被证明可以改善感知和行为判断，并加快对外部事件的反应，特别是那些被噪音严重污染或模糊的unisensory表征。为了实现这些好处，大脑必须协调具有不同操作参数的感官活动，特别是依赖于直接环境条件的不同可靠性。为了达到最佳的整合，大脑必须根据这些可靠性，在联合考虑中适当地“加权”来自不同感官的信号。这里提出的框架旨在理解这种现象，通过评估是否使用卡尔曼滤波器动态连续时间多感官任务，以实现最佳性能。受试者的任务是跟踪动态和噪声破坏模式的视觉和听觉刺激单独或在演唱会。主体适应信号变化的能力和感官的相对可靠性在所提出的模型框架的背景下进行解释。受试者的期望是由先前的经验和信号和信号统计变化的预警决定的，并且评估他们根据这些信息在响应中做出调整的时机和影响。在拟议的研究中应用的实验方法将提供重要的见解的原则后，感觉通道相结合，并帮助建立边界条件后，实现实时集成。