Interplay of amodal and modal encodings underlying directional space-metric associations

方向空间度量关联下的非模态和模态编码的相互作用

• 批准号: 422445168
• 负责人: Professor Dr. Martin Butz
• 金额: --
• 依托单位: Arbeitsgruppe Diagnostik und Kognitive Neuropsychologie
• 依托单位国家: 德国
• 项目类别: Research Units
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/422445168?language=en
• 关键词: Interplay; amodal; modal; encodings; underlying

The modal or amodal origin of directional space-metric associations (SMAs) – as revealed, for example, by the spatial-numerical associations of response codes (SNARC) effect – is currently under debate. Modal encodings imply culturally-grounded, long-term associations of particular stimuli with space – such as the mental number line, which extends from left to right in western cultures. Amodal encodings, on the other hand, refer to the manner how entities, including numbers, are temporally encoded ordinally in working memory (WM) – where the order is associated with a virtual, left-to-right axis in western cultures. The results from our joint project have shown that both long term associations (including number-line-like effects and distinct stimuli, such as the letter ‘A’ on the left side) and WM content play significant roles. Moreover, in our studies in virtual realities (VR), we have discovered that the spatial associations strongly depend on which spatial axes are task-relevant. Meanwhile, we have developed a resourceful event-predictive inference (REPI) system, which models the cognitive dynamics that unfold during the involved perceptual and decision making processes. REPI offers explicit computational and algorithmic explanations about how SMAs may affect stimulus processing and response behavior across modal and amodal representations. However, many open questions remain, particularly concerning the precise nature of the modal and amodal interactions, their selective task-relevant spatial associations, and the modularized modelling of the interaction dynamics. In the present proposal, we thus follow up on this prior work. In WP 1, we will examine the logic behind the dominant role of stimulus locations, hand arrangements, and response targets in further VR studies. We will also look at the (potential) limits of generalization of the amodal ordinal working memory (WM) process by using other ordinal stimuli – not only numbers and letters. In WP2, we will systematically investigate whether the role of WM changes as a function of individual stimuli, which would indicate that WM content is not as amodal as postulated. In WP3, we will further challenge the amodality hypothesis of WM content by systematically varying the modalities in the encoding and classification phase. Finally, in WP4, we will look at the developmental trajectories in a cross-sectional experiment to find out how modal and amodal encodings of SMAs develop over time. In all four work packages, the results (as well as previous and other available results) will be modelled with the REPI framework. We thus hope to be able to offer a complete computational and algorithmic explanation of SMAs and their effects on reaction times, error rates, and decision making dynamics both across trials and over development.

定向空间度量关联（SMA）的模态或非模态起源——例如通过响应代码的空间数字关联（SNARC）效应所揭示的——目前正在争论中。模态编码意味着特定刺激与空间的基于文化的长期关联——例如心理数轴，在西方文化中从左向右延伸。另一方面，非模态编码是指实体（包括数字）在工作记忆（WM）中按时间顺序编码的方式——其中顺序与西方文化中的虚拟的从左到右的轴相关联。我们的联合项目的结果表明，长期关联（包括类似数线的效应和独特的刺激，例如左侧的字母“A”）和 WM 内容都发挥着重要作用。此外，在虚拟现实（VR）研究中，我们发现空间关联很大程度上取决于哪些空间轴与任务相关。同时，我们开发了一个资源丰富的事件预测推理（REPI）系统，该系统对所涉及的感知和决策过程中展开的认知动态进行建模。 REPI 提供了关于 SMA 如何影响跨模态和非模态表征的刺激处理和响应行为的明确计算和算法解释。然而，仍然存在许多悬而未决的问题，特别是关于模态和非模态交互的精确性质、它们选择性的与任务相关的空间关联以及交互动力学的模块化建模。因此，在本提案中，我们对先前的工作进行了跟进。在 WP 1 中，我们将在进一步的 VR 研究中研究刺激位置、手部布置和反应目标的主导作用背后的逻辑。我们还将通过使用其他序数刺激（而不仅仅是数字和字母）来研究非模态序数工作记忆（WM）过程的（潜在）泛化限制。在 WP2 中，我们将系统地研究 WM 的作用是否会随着个体刺激的变化而变化，这表明 WM 内容并不像假设的那样是非模态的。在 WP3 中，我们将通过系统地改变编码和分类阶段的模态来进一步挑战 WM 内容的模态假说。最后，在 WP4 中，我们将通过横断面实验观察发育轨迹，以了解 SMA 的模态和非模态编码如何随着时间的推移而发展。在所有四个工作包中，结果（以及以前的和其他可用的结果）将使用 REPI 框架进行建模。因此，我们希望能够对 SMA 及其对整个试验和过度开发的反应时间、错误率和决策动态的影响提供完整的计算和算法解释。