Interplay of amodal and modal encodings underlying directional space-metric associations

定向空间-度量关联（sma）的模态或模态起源——如响应码的空间-数值关联（SNARC）效应所揭示的那样——目前正在争论中。模态编码暗示了特定刺激与空间的文化基础、长期联系——比如西方文化中从左向右延伸的心理数轴。另一方面，模态编码指的是包括数字在内的实体在工作记忆（WM）中暂时按顺序编码的方式——在西方文化中，这种顺序与一个虚拟的、从左到右的轴有关。我们联合项目的结果表明，长期关联（包括数字线效应和不同的刺激，如左侧的字母“A”）和WM内容都起着重要作用。此外，在我们对虚拟现实（VR）的研究中，我们发现空间关联强烈依赖于哪些空间轴与任务相关。同时，我们开发了一个资源丰富的事件预测推理（REPI）系统，该系统模拟了在涉及感知和决策过程中展开的认知动态。REPI提供了明确的计算和算法解释，说明sma如何影响跨模态和模态表征的刺激处理和反应行为。然而，许多悬而未决的问题仍然存在，特别是关于模态和模态相互作用的确切性质，它们的选择性任务相关的空间关联，以及相互作用动力学的模块化建模。因此，在本提案中，我们对这项先前的工作采取后续行动。在WP 1中，我们将在进一步的VR研究中研究刺激位置、手的排列和反应目标的主导作用背后的逻辑。我们还将通过使用其他有序刺激——不仅仅是数字和字母——来研究模态有序工作记忆（WM）过程泛化的（潜在）限制。在WP2中，我们将系统地研究WM的作用是否作为个体刺激的函数而变化，这将表明WM的内容并不像假设的那样是模态的。在WP3中，我们将通过系统地改变编码和分类阶段的模态，进一步挑战WM内容的模态假设。最后，在WP4中，我们将在横断面实验中观察发育轨迹，以了解sma的模态和模态编码如何随时间发展。在所有四个工作包中，结果（以及之前的和其他可用的结果）将使用REPI框架进行建模。因此，我们希望能够对sma及其对反应时间、错误率和决策动态的影响提供完整的计算和算法解释。

The modal or amodal origin of directional space-metric associations (SMAs) – as revealed, for example, by the spatial-numerical associations of response codes (SNARC) effect – is currently under debate. Modal encodings imply culturally-grounded, long-term associations of particular stimuli with space – such as the mental number line, which extends from left to right in western cultures. Amodal encodings, on the other hand, refer to the manner how entities, including numbers, are temporally encoded ordinally in working memory (WM) – where the order is associated with a virtual, left-to-right axis in western cultures. The results from our joint project have shown that both long term associations (including number-line-like effects and distinct stimuli, such as the letter ‘A’ on the left side) and WM content play significant roles. Moreover, in our studies in virtual realities (VR), we have discovered that the spatial associations strongly depend on which spatial axes are task-relevant. Meanwhile, we have developed a resourceful event-predictive inference (REPI) system, which models the cognitive dynamics that unfold during the involved perceptual and decision making processes. REPI offers explicit computational and algorithmic explanations about how SMAs may affect stimulus processing and response behavior across modal and amodal representations. However, many open questions remain, particularly concerning the precise nature of the modal and amodal interactions, their selective task-relevant spatial associations, and the modularized modelling of the interaction dynamics. In the present proposal, we thus follow up on this prior work. In WP 1, we will examine the logic behind the dominant role of stimulus locations, hand arrangements, and response targets in further VR studies. We will also look at the (potential) limits of generalization of the amodal ordinal working memory (WM) process by using other ordinal stimuli – not only numbers and letters. In WP2, we will systematically investigate whether the role of WM changes as a function of individual stimuli, which would indicate that WM content is not as amodal as postulated. In WP3, we will further challenge the amodality hypothesis of WM content by systematically varying the modalities in the encoding and classification phase. Finally, in WP4, we will look at the developmental trajectories in a cross-sectional experiment to find out how modal and amodal encodings of SMAs develop over time. In all four work packages, the results (as well as previous and other available results) will be modelled with the REPI framework. We thus hope to be able to offer a complete computational and algorithmic explanation of SMAs and their effects on reaction times, error rates, and decision making dynamics both across trials and over development.