CHS: Medium: Collaborative Research: Increasing Communication Rates through a TActile Phonemic Sleeve (TAPS)

通过皮肤能传递多少信息？传统观点认为，视觉和听觉的作用不如视觉和听觉大。然而，研究人员认为，有必要更好地了解如何用皮肤自己的语言与皮肤“对话”。该团队最近设计了一个带有24个振动器的套筒，可以以不同的振动模式传递语音。对100多名听力正常的人（除了一名听力严重受损的参与者）的测试结果表明，基于振动的语音表示可以很容易地学习，并结合起来单独“读出”英语单词。最好的学习者已经证明了他们每分钟学习一个英语单词的能力，最多可以掌握500个英语单词。这是一个令人兴奋的突破，可能会为严重听力障碍的人以及因嘈杂环境而听力暂时受损的人（如消防员）带来一种通信设备。该项目的主要目的是系统地探索通过套筒提高通信速率的方法。我们的目标是达到每分钟至少60-80个单词，这个交流速度已经被Tadoma语言交流方法所证明，在这种交流方法中，聋哑人和盲人把一只手放在说话人的脸和脖子上，以理解说话人的语言。这样的交流速率将使袖成为一个实用的解决方案，为社区的人谁需要另一种方式的语言交流，如那些有听力和视力障碍。如果成功，该项目将为这个弱势社区提供更好的生活质量（在社会互动和就业能力方面）。该项目解决了在通过语音触觉显示提高通信速率的背景下，触觉信息容量的基本问题。该项目利用了该团队在以下方面的研究和开发经验：(i)聋哑人和聋哑人盲人使用的自然语言交流方法；（ii）可穿戴触觉技术；（iii）评估人的表现的信息理论方法；（四）触觉的心理物理特性。该项目建立在研究人员最近的成功基础上，他们展示了为英语中的39个音素开发触觉代码的能力，这些音素在单独和由音素序列组成的单词中都是高度可识别的。这项研究的重点是提高经过训练的参与者通过触摸传递信息的速度，这些参与者包括听力正常的观察者，以及代表该设备潜在最终用户的主要类别的感觉神经性听力障碍患者。总体目标是实现触觉语音刺激的传输速率，与使用Tadoma方法的聋哑用户的传输速率相当，并采用有效的方法训练显示器用户以高精度接收连接的语音信号。实验将对触觉系统的使用者进行训练和测试，测试他们识别音素、单词、短语和对话句子的能力。这些实验将探讨刺激的持续特性，目的是建立音素持续时间的下限以及音素与单词之间的间隔。有效通信速率的估计将考虑到传输速率和接收的准确性，其目标是显示每分钟50-100个单词的有效通信速率。以每秒比特数为单位的触觉通信速率的下限估计可以从本项目结束时实现的传输速率中得出。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

How much information can be transmitted through the skin? Conventional wisdom would say not as much as vision or hearing. The researchers argue, however, that there is a need to better understand how to "talk" to the skin in its own language. The team has recently devised a sleeve with 24 vibrators that delivers speech sounds in the form of distinct vibration patterns. Testing results from more than 100 normal-hearing individuals (in addition to one participant with a severe hearing loss) show that the vibration-based representation of speech sounds can be easily learned and combined to "sound out" English words in isolation. The best learners have demonstrated an ability to learn one English word per minute, up to a vocabulary size of 500 English words. This is an exciting breakthrough that can lead to a communication device for people with severe hearing impairments and for those whose hearing is temporarily compromised due to noisy environments such as firefighters. The main objective of this project is to systematically explore ways that the communication rate through the sleeve can be increased. Our goal is to reach at least 60-80 words per minute, a communication rate that has been demonstrated by the Tadoma method of speech communication, in which persons who are both deaf and blind place a hand over the face and neck of the talker to understand speech. Such a communication rate will make the sleeve a practical solution for the community of people who need an alternate means for speech communication, such as the ones with both hearing and vision impairments. If successful, the project will facilitate a better quality of life (in terms of social interactions as well as employability) for this disadvantaged community.This project addresses the fundamental question of the information capacity of the sense of touch within the context of increasing communication rates through a haptic display of speech. The project leverages the team's research and development experience on: (i) natural speech communication methods used by the deaf and deaf-blind; (ii) wearable haptic technologies; (iii) an information-theoretical approach to assessing human performance; and (iv) psychophysical properties of the tactual sense. The project builds upon the researchers' recent success at demonstrating their ability to develop tactile codes for the 39 phonemes of English that were highly recognizable in isolation and in words formed from sequences of phonemes. The research is focused on increasing the rate at which information can be transmitted through touch by trained participants that include observers with normal hearing as well as those with sensorineural hearing impairments who represent a major category of potential end users of the device. The overall goal is to achieve transmission rates of tactile speech stimuli that are comparable to those achieved by deaf-blind users of the Tadoma method, and to employ effective methods of training users of the display for reception of connected speech signals with a high degree of accuracy. Experiments will be carried out in which users of the tactile system are trained and tested on their ability to recognize phonemes, words, short phrases, and finally conversational sentences. Durational properties of the stimuli will be explored in these experiments with the aim of establishing the lower limits on phoneme duration and the intervals between phonemes and words. Estimates of effective communication rates will take into account both the transmission rate and accuracy of reception, with the goal of demonstrating effective communication rates on the order of 50-100 words per minute. A lower-bound estimate for haptic communication rate in terms of bits per second can then be derived from the transmission rate achieved at the end of this project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.