Interplay of amodal and modal encodings underlying directional space-metric associations

方向性空间度量关联（SMA）的模态或非模态起源-例如，由响应代码的空间-数值关联（SNARC）效应所揭示的-目前正在辩论中。模态编码意味着特定刺激与空间的文化基础的长期关联-例如心理数字线，在西方文化中从左到右延伸。另一方面，非模态编码指的是实体（包括数字）在工作记忆（WM）中按时间顺序编码的方式-在西方文化中，顺序与虚拟的从左到右的轴相关联。我们合作项目的结果表明，长时关联（包括数字线效应和不同的刺激，如左侧的字母“A”）和工作记忆内容都起着重要作用。此外，在虚拟现实（VR）的研究中，我们发现空间关联强烈依赖于哪些空间轴与任务相关。同时，我们开发了一个足智多谋的事件预测推理（REPI）系统，该系统模拟了在所涉及的感知和决策过程中展现的认知动态。REPI提供了明确的计算和算法解释SMA如何影响刺激处理和反应行为的模态和非模态表征。然而，许多悬而未决的问题仍然存在，特别是关于模态和非模态相互作用的确切性质，其选择性的任务相关的空间协会，以及模块化的互动动态建模。因此，在本提案中，我们对先前的工作采取了后续行动。在WP 1中，我们将在进一步的VR研究中研究刺激位置，手部排列和反应目标的主导作用背后的逻辑。我们还将通过使用其他序数刺激（不仅仅是数字和字母）来研究非模态序数工作记忆（WM）过程的（潜在）泛化限制。在WP 2中，我们将系统地研究WM的作用是否会随着个体刺激的变化而变化，这将表明WM内容并不像假设的那样是非模态的。在WP 3中，我们将通过系统地改变编码和分类阶段的模式来进一步挑战WM内容的无模式假设。最后，在WP 4中，我们将在一个横截面实验中研究SMA的发展轨迹，以了解SMA的模态和非模态编码如何随着时间的推移而发展。在所有四个工作包中，结果（以及以前和其他现有的结果）将采用REPI框架进行建模。因此，我们希望能够提供一个完整的计算和算法解释SMA及其对反应时间，错误率和决策动态的影响，无论是在试验和开发。

The modal or amodal origin of directional space-metric associations (SMAs) – as revealed, for example, by the spatial-numerical associations of response codes (SNARC) effect – is currently under debate. Modal encodings imply culturally-grounded, long-term associations of particular stimuli with space – such as the mental number line, which extends from left to right in western cultures. Amodal encodings, on the other hand, refer to the manner how entities, including numbers, are temporally encoded ordinally in working memory (WM) – where the order is associated with a virtual, left-to-right axis in western cultures. The results from our joint project have shown that both long term associations (including number-line-like effects and distinct stimuli, such as the letter ‘A’ on the left side) and WM content play significant roles. Moreover, in our studies in virtual realities (VR), we have discovered that the spatial associations strongly depend on which spatial axes are task-relevant. Meanwhile, we have developed a resourceful event-predictive inference (REPI) system, which models the cognitive dynamics that unfold during the involved perceptual and decision making processes. REPI offers explicit computational and algorithmic explanations about how SMAs may affect stimulus processing and response behavior across modal and amodal representations. However, many open questions remain, particularly concerning the precise nature of the modal and amodal interactions, their selective task-relevant spatial associations, and the modularized modelling of the interaction dynamics. In the present proposal, we thus follow up on this prior work. In WP 1, we will examine the logic behind the dominant role of stimulus locations, hand arrangements, and response targets in further VR studies. We will also look at the (potential) limits of generalization of the amodal ordinal working memory (WM) process by using other ordinal stimuli – not only numbers and letters. In WP2, we will systematically investigate whether the role of WM changes as a function of individual stimuli, which would indicate that WM content is not as amodal as postulated. In WP3, we will further challenge the amodality hypothesis of WM content by systematically varying the modalities in the encoding and classification phase. Finally, in WP4, we will look at the developmental trajectories in a cross-sectional experiment to find out how modal and amodal encodings of SMAs develop over time. In all four work packages, the results (as well as previous and other available results) will be modelled with the REPI framework. We thus hope to be able to offer a complete computational and algorithmic explanation of SMAs and their effects on reaction times, error rates, and decision making dynamics both across trials and over development.