HCC: Small: Passive Tactile Learning and Rehabilitation Using Wearable Computers

HCC：小型：使用可穿戴计算机进行被动触觉学习和康复

• 批准号: 1217473
• 负责人: Thad Starner
• 金额: $ 50万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1217473&HistoricalAwards=false
• 关键词: HCC; Small; Passive; Tactile; Learning

Mobile Music Touch (MMT), developed in prior work by the PI and his colleagues, is a lightweight, wireless tactile music instruction system consisting of fingerless gloves and a Bluetooth-enabled mobile phone. Piano passages to be learned are loaded into the mobile phone and played repeatedly in the user's earpiece while s/he performs other tasks. As each note of the song plays, a vibrator on the appropriate finger in the gloves activates, indicating which finger is used to play the note. MMT users showed significant improvement in their ability to reproduce simple piano melodies, even though they were actively engaged in a reading comprehension task during practice. The PI terms this effect Passive Tactile Learning (PTL). He also has initial evidence for a related effect, Passive Tactile Rehabilitation (PTR), where tactile stimulation of the hand correlates to improvements in dexterity and sensation for people with tetraplegia resulting from incomplete spinal cord injury. In this project, the PI will explore PTL/PTR effects in a variety of applications, develop procedural and device design guidelines to best elicit these effects, and illustrate methods and metrics that are sensitive to detecting these effects.Using MMT, he will compare users' learning of piano melodies while being distracted by a math-intensive test in four conditions: vibration stimulation with accompanying audio, vibration alone, audio alone, and no stimulation. This study will quantify the PTL effect and establish if vibration alone is sufficient to cause a learning effect. In an effort to broaden the applicability of the method, he will study the use of PTL to aid in learning stenographic typing and dance steps. To explore the possibility of PTR, he will run a 12-18 person controlled study with participants with tetraplegia due to partial spinal cord injury. This "in-the-wild" PTL/PTR study will compare active practice on a piano with active practice augmented with MMT. Participants will wear the glove during their everyday lives for at least 2 hours a day. The PI will compare learning rates as well as changes in participants' scores on standard sensation and dexterity tests. And he will also conduct a wearability study on MMT for persons with spinal cord injury to better accommodate their needs.Broader Impacts: Establishing guidelines for the use and effectiveness of PTL and PTR will enable others to apply the concepts in different domains, e.g., learning sign language or manual procedures such as a pre-flight checklist or training users of prosthetic limbs in how to trigger different actions in their limb. Beyond partial spinal cord injury, PTR might also be exploited for recovery from damage from stroke or lesions due to Multiple Sclerosis.The PI maintains an aggressive outreach program. Beyond professional publication of results in both the Human Computer Interaction and Rehabilitation literature, he will leverage his contacts to present the work in mainstream press and media (e.g., CNN), as well as more niche communities such as the Georgia Radio Relay Service. One of his focuses is exposing children in Georgia area schools for the deaf to his research, as there is a distinct need for encouraging deaf children in STEM areas. This effort will also be part of the undergraduate "Device Thread" in Georgia Tech's College of Computing, which exposes undergraduates to the rigors of designing computing for situations with physical device limitations, such as mobile phones, wearable devices, and robotics. Graduates and undergraduates in these classes will help investigate and fabricate new PTL/PTR devices. Finally, the PI intends to use this effort in his Computing4Good campaign, which inspires students to think of computing as a means for enabling social change.

移动音乐触摸（MMT）是由PI和他的同事在之前的工作中开发的，是一种轻量级的无线触觉音乐教学系统，由无指手套和蓝牙手机组成。要学习的钢琴段落被加载到手机中，在用户执行其他任务时，在耳机中反复播放。当歌曲的每个音符播放时，手套中相应手指上的振动器就会被激活，指示哪个手指是用来播放这个音符的。即使在练习中积极参与阅读理解任务，MMT使用者在重现简单钢琴旋律的能力上也有显著提高。PI将这种效应称为被动触觉学习（PTL）。他也有一个相关效果的初步证据，被动触觉康复（PTR），其中手部的触觉刺激与由不完全性脊髓损伤引起的四肢瘫痪患者的灵活性和感觉的改善相关。在这个项目中，PI将探索各种应用中的PTL/PTR效应，制定程序和设备设计指南，以最好地引出这些效应，并说明检测这些效应的敏感方法和指标。利用MMT，他将在四种情况下比较用户在被数学密集测试分心的情况下学习钢琴旋律的情况：有音频的振动刺激、只有振动、只有音频和没有刺激。本研究将量化PTL效应，并确定振动本身是否足以引起学习效应。为了扩大该方法的适用性，他将研究如何使用PTL来帮助学习速记打字和舞蹈步骤。为了探索PTR的可能性，他将进行一项12-18人的对照研究，参与者都是由于部分脊髓损伤而四肢瘫痪的。这个“野外”PTL/PTR研究将比较钢琴上的主动练习和辅以MMT的主动练习。参与者将在日常生活中每天佩戴手套至少2小时。PI将比较学习速度以及参与者在标准感觉和敏捷测试中的得分变化。他还将为脊髓损伤患者进行MMT的可穿戴性研究，以更好地满足他们的需求。更广泛的影响：为PTL和PTR的使用和有效性建立指导方针，将使其他人能够将这些概念应用于不同的领域，例如，学习手语或手动程序，如飞行前检查表，或培训假肢用户如何触发肢体的不同动作。除了部分脊髓损伤，PTR也可用于从中风或多发性硬化症引起的损伤中恢复。PI保持着积极的外展计划。除了在人机交互和康复文献中发表专业成果外，他还将利用他的人脉在主流媒体和媒体（如CNN）以及更多的利基社区（如Georgia Radio Relay Service）上展示他的工作。他的重点之一是让乔治亚地区的聋哑学校的孩子们了解他的研究，因为在STEM领域，鼓励聋哑儿童的需求非常明显。这项工作也将成为佐治亚理工学院计算学院本科生“设备线程”的一部分，该项目将使本科生接触到为具有物理设备限制的情况（如移动电话、可穿戴设备和机器人）设计计算的严格性。这些课程的毕业生和本科生将帮助研究和制造新的PTL/PTR器件。最后，PI打算在他的Computing4Good活动中利用这一努力，激励学生将计算视为实现社会变革的一种手段。