Temporal ventriloquism effect and auditory dominance in multimodal duration perception

如果一个事件同时刺激了几种模态，那么来自不同模态的输入必须被整合成一个单一的、连贯的感知印象。只有少数以前的研究调查了多感觉整合过程中的时间知觉域。这些研究表明，从不同的方式持续时间信息之间的冲突通常有利于解决的听觉信息。在本研究项目的第一阶段，我们扩展了现有的证据表明，听觉优势的多感官持续时间整合，虽然在较小程度上，视觉信息也有助于合并的时间感知。此外，我们提供的证据表明，这些多感觉整合的影响可能是由颞腹语介导的，是由于一个短暂的，而不是一个持续的机制。有趣的是，感知持续时间也会受到空间信息的影响，而空间信息最可靠地由视觉模态传达。我们发现，这种视觉空间信息强烈影响感知听觉持续时间。因此，感觉优势并不是与生俱来的，而是沿着环境因素而变化的。也就是说，当来自不同模态的信息存在时，一种感觉驱动整体感知的程度取决于多个感知域（如空间和时间）之间的相互作用。为了扩展这些结果，我们提出了一个新的系列14个实验，解决五个研究目标：（A）需要进一步的对照研究，以加强我们的结论，多模态时间感知是短暂的，而不是持续的。(B)到目前为止，多模态时距整合的研究仅限于听觉和视觉刺激。在一系列的实验中，这些结果将被推广到包括触觉模态在内的多模态交互。一方面，这将拓宽我们的研究范围，另一方面，希望为目标（C）的一系列实验提供一个有用的先决条件：最优集成的概念。具体来说，这一概念认为，来自不同模态的贡献根据其感官输入的可靠性进行加权，以最大限度地提高感知系统的灵敏度。最后，关于时间的多模态整合的一个关键方面是调查这个过程的轨迹。多通道整合效应可能反映了来自不同通道的输入之间真正的感觉相互作用，但也可能出现在稍后的决策阶段。我们提出了两个系列的实验，以解决这个理论上的关键问题，采用（D）行为和（E）电生理措施。这些实验将有望提供新的和有价值的见解的原则和机制的多模态整合的感知持续时间。

If an event stimulates several modalities at the same time, the inputs from the different modalities have to be integrated into a single and coherent perceptual impression. Only few previous studies have investigated multisensory integration processes in the domain of time perception. These studies indicate that conflicts between duration information from different modalities are typically resolved in favour of the auditory information. In the first phase of this research project, we extended this existing evidence about auditory dominance in multisensory duration integration by demonstrating that - albeit to a lesser extent - the visual information also contributes to the combined temporal percept. In addition, we provided evidence that these multisensory integration effects are likely mediated by temporal ventriloquism, and are due to a transient rather than a sustained mechanism. Interestingly, perceived duration can also be affected by spatial information, which is most reliably conveyed by the visual modality. We showed that such visuo-spatial information strongly affects perceived auditory duration. Thus, sensory dominance is not hard-wired, but varies along with contextual factors. That is, when information from different modalities is present, the extent to which one sense drives the overall percept depends on the interaction between multiple perceptual domains, such as space and time. To extend these results, we suggest a novel series of 14 experiments addressing five research objectives: (A) a further control study is needed to strengthen our conclusion that multimodal time perception is transient rather than sustained. (B) So far, multimodal duration integration was investigated exclusively with auditory and visual stimuli. In a series of experiments, these results will be generalized to multimodal interactions including the tactile modality. On the one hand, this will broaden the scope of our research, and on the other hand, hopefully provide a useful prerequisite for the series of experiments on objective (C): the notion of optimal integration. Specifically, this notion holds that the contributions from different modalities are weighted according to the reliability of their sensory input, in order to maximise the sensitivity of the perceptual system. Finally, a crucial aspect concerning the multimodal integration of time is to investigate the locus of this process. Multimodal integration effects might reflect genuine sensory interactions between the inputs from different modalities, but alternatively, they might emerge later, on a decisional stage of processing. We propose two series of experiments to address this theoretically pivotal issue by employing (D) behavioural and (E) electrophysiological measures. These experiments will hopefully provide novel and valuable insights in the principles and mechanisms underlying the multimodal integration of perceived duration.