Visuo-haptic object processing, memory representations and multisensory integration

点击翻译按钮获取中文摘要

How does your mind connect the sensation of an object in your hand with the sight of the same object? Why do you experience a single object and not two? If you hold a key, the key you feel in your hand and the one you see are perceived as a single object. We do this seamlessly, without having to think about it, and we are able to recognize the key when we see it and when we touch it. Yet, much of the research done on how we recognize objects has focused mostly recognizing objects by sight, and not by touch. Also, recent research that has looked at integrating the senses has done so using complex objects for which it is difficult to understand how objects differ or the kind of information people use when recognizing the object. My research program uses novel objects that vary on fixed dimensions - we can therefore tell how the objects differ, and what information people use when recognizing them. I plan to use these novel objects to study three important questions about the information we keep in memory about objects and how we recognize objects when we see or touch them. The first question is: what kind of information do we store in memory? Is it a visual image of the object or a verbal description of the object? To answer that question, I will ask people to learn to recognize new objects, but I will also interfere with their learning by presenting additional information. The key here is that if the information I present is similar to the information people are using to create object memories, then it will be more difficult to create these memories. The second question is: how do we use our sense of vision and touch when we create these memories? Is one sense more important than the other? To answer that question, I will ask people to learn to recognize objects by looking at them, by touching them, or by looking and touching objects at the same time. If one sense is more important than the other senses for creating memories, people will be better at completing the experiment when using that sense. Finally, the third question is: is vision more important than touch when identifying objects? To answer this question, I will ask people to identify objects by sight or by touch, but present another object to the other sense at the same time - people are supposed to ignore that second object. If the `second sense' is important, people will not be able to ignore that second object, and they may be slower and less accurate when identifying objects. My research has important implications and applications. It will help us learn the kind of information that memories contain, and how we create and access these memories. This information will also be helpful when we design technology like cell phones that incorporate sound (a ringtone), sight (the screen), and touch (a vibration). Finally, my research may help clinicians who are helping others overcome sensory loss like blindness, by letting them know how they can use other senses.