Visuo-haptic object processing, memory representations and multisensory integration

你的大脑是如何将你手中的物体的感觉与同样物体的视觉联系起来的？为什么你只体验到一个物体而不是两个？如果你拿着一把钥匙，你感觉到的那把钥匙和你看到的那把钥匙会被视为一个单一的物体。我们做到了天衣无缝，不需要思考，我们能够识别钥匙，当我们看到它，当我们触摸它。然而，许多关于我们如何识别物体的研究主要集中在通过视觉识别物体，而不是通过触摸。此外，最近关于整合感官的研究使用了复杂的物体，很难理解物体的不同之处，也很难理解人们在识别物体时使用的信息类型。我的研究项目使用在固定维度上变化的新物体，因此我们可以分辨出物体的不同之处，以及人们在识别它们时使用的信息。我计划用这些新奇的物体来研究三个重要的问题，关于我们在记忆中保存的关于物体的信息，以及当我们看到或触摸物体时，我们是如何识别它们的。第一个问题是：我们在记忆中存储了什么样的信息？它是物体的视觉形象还是对物体的口头描述？为了回答这个问题，我会要求人们学会识别新物体，但我也会通过提供额外的信息来干扰他们的学习。这里的关键是，如果我提供的信息与人们用来创建对象记忆的信息相似，那么创建这些记忆就会更加困难。第二个问题是：当我们创造这些记忆时，我们如何使用我们的视觉和触觉？一种感觉比另一种更重要吗？为了回答这个问题，我会让人们学会通过看物体，触摸物体，或者同时看和触摸物体来识别物体。如果一种感官在创造记忆方面比其他感官更重要，那么人们在使用这种感官时将更好地完成实验。最后，第三个问题是：在识别物体时，视觉比触觉更重要吗？为了回答这个问题，我会让人们通过视觉或触觉来识别物体，但同时向另一种感官呈现另一个物体，人们应该忽略第二个物体。如果第二感觉很重要，人们将无法忽视第二个物体，他们在识别物体时可能会变慢，也不那么准确。我的研究具有重要的意义和应用。它将帮助我们了解记忆所包含的信息，以及我们如何创造和访问这些记忆。这些信息在我们设计诸如手机这样集声音（铃声）、视觉（屏幕）和触摸（振动）于一体的技术时也很有帮助。最后，我的研究可能会帮助那些帮助其他人克服失明等感官丧失的临床医生，让他们知道如何使用其他感官。

How does your mind connect the sensation of an object in your hand with the sight of the same object? Why do you experience a single object and not two? If you hold a key, the key you feel in your hand and the one you see are perceived as a single object. We do this seamlessly, without having to think about it, and we are able to recognize the key when we see it and when we touch it. Yet, much of the research done on how we recognize objects has focused mostly recognizing objects by sight, and not by touch. Also, recent research that has looked at integrating the senses has done so using complex objects for which it is difficult to understand how objects differ or the kind of information people use when recognizing the object. My research program uses novel objects that vary on fixed dimensions we can therefore tell how the objects differ, and what information people use when recognizing them. I plan to use these novel objects to study three important questions about the information we keep in memory about objects and how we recognize objects when we see or touch them. The first question is: what kind of information do we store in memory? Is it a visual image of the object or a verbal description of the object? To answer that question, I will ask people to learn to recognize new objects, but I will also interfere with their learning by presenting additional information. The key here is that if the information I present is similar to the information people are using to create object memories, then it will be more difficult to create these memories. The second question is: how do we use our sense of vision and touch when we create these memories? Is one sense more important than the other? To answer that question, I will ask people to learn to recognize objects by looking at them, by touching them, or by looking and touching objects at the same time. If one sense is more important than the other senses for creating memories, people will be better at completing the experiment when using that sense. Finally, the third question is: is vision more important than touch when identifying objects? To answer this question, I will ask people to identify objects by sight or by touch, but present another object to the other sense at the same time people are supposed to ignore that second object. If the second sense' is important, people will not be able to ignore that second object, and they may be slower and less accurate when identifying objects. My research has important implications and applications. It will help us learn the kind of information that memories contain, and how we create and access these memories. This information will also be helpful when we design technology like cell phones that incorporate sound (a ringtone), sight (the screen), and touch (a vibration). Finally, my research may help clinicians who are helping others overcome sensory loss like blindness, by letting them know how they can use other senses.