Visuo-haptic object processing, memory representations and multisensory integration

你的大脑是如何将你手中物体的感觉与同一物体的视觉联系起来的？为什么你只体验一个物体，而不是两个？如果你拿着一把钥匙，你手中感觉到的钥匙和你看到的那把钥匙都会被感知为一个单一的物体。我们无缝地做到了这一点，不必考虑它，当我们看到它时，当我们触摸它时，我们能够识别出它。然而，许多关于我们如何识别物体的研究主要集中在通过视觉识别物体，而不是通过触觉识别物体。此外，最近着眼于整合感官的研究使用了复杂的物体，这些物体很难理解物体的不同之处，也很难理解人们在识别物体时使用的信息类型。我的研究项目使用的是在固定维度上变化的新奇物体--因此，我们可以分辨出这些物体的不同之处，以及人们在识别它们时使用的信息。我计划用这些新奇的物体来研究三个重要的问题，即我们记忆中关于物体的信息，以及当我们看到或触摸物体时，我们是如何识别它们的。第一个问题是：我们在内存中存储了什么样的信息？它是物体的视觉形象，还是对物体的口头描述？为了回答这个问题，我会让人们学习识别新的物体，但我也会通过提供额外的信息来干扰他们的学习。这里的关键是，如果我展示的信息与人们用来创造物体记忆的信息相似，那么创造这些记忆就会更加困难。第二个问题是：当我们创造这些记忆时，我们如何使用我们的视觉和触觉？其中一种感觉比另一种更重要吗？为了回答这个问题，我会让人们通过看物体、触摸物体或同时看和触摸物体来学习识别物体。如果其中一种感觉在创造记忆方面比其他感觉更重要，人们在使用这一感觉时会更好地完成实验。最后，第三个问题是：在识别物体时，视觉比触觉更重要吗？为了回答这个问题，我会让人们通过视觉或触觉来识别物体，但同时将另一个物体呈现给另一种感觉--人们应该忽略第二个物体。如果第二感觉很重要，人们将无法忽视第二个物体，他们在识别物体时可能会变得更慢、更不准确。我的研究具有重要的意义和应用价值。它将帮助我们了解记忆包含的信息种类，以及我们如何创建和访问这些记忆。当我们设计像手机这样集声音(铃声)、视觉(屏幕)和触摸(振动)于一体的技术时，这些信息也将是有帮助的。最后，我的研究可能会帮助那些正在帮助他人克服失明等感觉丧失的临床医生，让他们知道如何使用其他感官。

How does your mind connect the sensation of an object in your hand with the sight of the same object? Why do you experience a single object and not two? If you hold a key, the key you feel in your hand and the one you see are perceived as a single object. We do this seamlessly, without having to think about it, and we are able to recognize the key when we see it and when we touch it. Yet, much of the research done on how we recognize objects has focused mostly recognizing objects by sight, and not by touch. Also, recent research that has looked at integrating the senses has done so using complex objects for which it is difficult to understand how objects differ or the kind of information people use when recognizing the object. My research program uses novel objects that vary on fixed dimensions - we can therefore tell how the objects differ, and what information people use when recognizing them. I plan to use these novel objects to study three important questions about the information we keep in memory about objects and how we recognize objects when we see or touch them. The first question is: what kind of information do we store in memory? Is it a visual image of the object or a verbal description of the object? To answer that question, I will ask people to learn to recognize new objects, but I will also interfere with their learning by presenting additional information. The key here is that if the information I present is similar to the information people are using to create object memories, then it will be more difficult to create these memories. The second question is: how do we use our sense of vision and touch when we create these memories? Is one sense more important than the other? To answer that question, I will ask people to learn to recognize objects by looking at them, by touching them, or by looking and touching objects at the same time. If one sense is more important than the other senses for creating memories, people will be better at completing the experiment when using that sense. Finally, the third question is: is vision more important than touch when identifying objects? To answer this question, I will ask people to identify objects by sight or by touch, but present another object to the other sense at the same time - people are supposed to ignore that second object. If the `second sense' is important, people will not be able to ignore that second object, and they may be slower and less accurate when identifying objects. My research has important implications and applications. It will help us learn the kind of information that memories contain, and how we create and access these memories. This information will also be helpful when we design technology like cell phones that incorporate sound (a ringtone), sight (the screen), and touch (a vibration). Finally, my research may help clinicians who are helping others overcome sensory loss like blindness, by letting them know how they can use other senses.