Can multisensory training aid visual learning? A computational investigation
Can multisensory training aid visual learning? A computational investigation
复制标题
多感官训练可以帮助视觉学习吗？
DOI：
10.1167/19.11.1
复制
发表时间：
2019
影响因子：
1.8
通讯作者：
Xu, Chenliang
中科院分区：
文献类型：
--
作者：
Jacobs, Robert A.;Xu, Chenliang
尽管现实世界的环境通常是多感官的，但视觉科学家通常在仅包含视觉信号的单感官环境中研究视觉学习。在这里，我们使用深度或人工神经网络来解决这个问题：多感官训练可以帮助视觉学习吗？我们在两种情况下基于视觉信号检查网络对对象的内部表示：(a) 当网络最初使用视觉和触觉信号进行训练时，以及 (b) 当网络最初仅使用视觉信号进行训练时。我们的结果表明，在视觉触觉环境（其中视觉信号而非触觉信号与方向相关）中训练的网络倾向于学习包含有用抽象的视觉表示，例如对象的分类结构，并且还学习对与对象识别或分类任务无关的成像参数（例如视点或方向）不太敏感的表示。我们的结论是，在仅视觉环境中研究感知学习的研究人员可能高估了与重要感知学习问题相关的困难。尽管多感官知觉有其自身的挑战，但当在多感官环境中考虑时，知觉学习会变得更容易。
Although real-world environments are often multisensory, visual scientists typically study visual learning in unisensory environments containing visual signals only. Here, we use deep or artificial neural networks to address the question, Can multisensory training aid visual learning? We examine a network's internal representations of objects based on visual signals in two conditions:(a) when the network is initially trained with both visual and haptic signals, and (b) when it is initially trained with visual signals only. Our results demonstrate that a network trained in a visual-haptic environment (in which visual, but not haptic, signals are orientation-dependent) tends to learn visual representations containing useful abstractions, such as the categorical structure of objects, and also learns representations that are less sensitive to imaging parameters, such as viewpoint or orientation, that are irrelevant for object recognition or classification tasks. We conclude that researchers studying perceptual learning in vision-only contexts may be overestimating the difficulties associated with important perceptual learning problems. Although multisensory perception has its own challenges, perceptual learning can become easier when it is considered in a multisensory setting.