HCC: Small: Passive Tactile Learning and Rehabilitation Using Wearable Computers

移动音乐触摸(MMT)是PI和他的同事在之前的工作中开发的，是一个轻量级的无线触觉音乐教学系统，由无手指手套和一部支持蓝牙的手机组成。要学习的钢琴段落被加载到手机中，在S/他执行其他任务的同时，在用户的耳机上重复播放。当播放歌曲的每个音符时，手套中相应手指上的振动器被激活，指示使用哪个手指来演奏音符。尽管MMT使用者在练习中积极地参与了一项阅读理解任务，但他们再现简单钢琴旋律的能力却有了显著的提高。PI将这种影响称为被动触觉学习(PTL)。他还有初步证据证明了被动触觉康复(PTR)的相关效应，在被动触觉康复中，对手的触觉刺激与因脊髓不全损伤而导致四肢瘫痪患者的灵巧度和感觉的改善有关。在这个项目中，PI将探索各种应用中的PTL/PTR效果，开发程序和设备设计指南来最好地获得这些效果，并说明对检测这些效果敏感的方法和指标。他将使用MMT，比较用户在四种情况下被数学密集测试分心时学习钢琴旋律的情况：伴随音频的振动刺激、单独振动、单独音频和没有刺激。这项研究将量化PTL效应，并确定单靠振动是否足以引起学习效应。为了扩大这种方法的适用性，他将研究如何使用PTL来帮助学习速记打字和舞步。为了探索PTR的可能性，他将对因部分脊髓损伤而四肢瘫痪的参与者进行一项12-18人的对照研究。这项“野外”的PTL/PTR研究将比较在钢琴上的积极练习和使用MMT增强的积极练习。参与者将在日常生活中戴上手套，每天至少两个小时。PI将比较学习速度以及参与者在标准感觉和灵活性测试中的分数变化。他还将为脊髓损伤患者进行MMT的佩戴性研究，以更好地满足他们的需求。广泛影响：为PTL和PTR的使用和有效性建立指南将使其他人能够在不同领域应用这些概念，例如学习手语或手动程序，如飞行前清单，或培训假肢用户如何触发其肢体的不同动作。除了部分脊髓损伤外，PTR也可用于中风或多发性硬化症损伤的恢复。PI保持积极的外展计划。除了在人机交互和康复文献中发表专业成果外，他还将利用他的联系人在主流媒体和媒体(如CNN)以及更多利基社区(如佐治亚州无线电转播服务)中介绍这项工作。他的重点之一是让佐治亚州地区聋人学校的孩子们了解他的研究，因为STEM地区显然需要鼓励聋人儿童。这项工作也将是佐治亚理工学院计算学院本科生“设备线程”的一部分，该课程让本科生接触到针对手机、可穿戴设备和机器人等物理设备限制的情况下设计计算的严酷要求。这些课程的毕业生和本科生将帮助研究和制造新的PTL/PTR器件。最后，PI打算在他的Computing4Good活动中利用这一努力，该活动激励学生将计算机视为实现社会变革的一种手段。

Mobile Music Touch (MMT), developed in prior work by the PI and his colleagues, is a lightweight, wireless tactile music instruction system consisting of fingerless gloves and a Bluetooth-enabled mobile phone. Piano passages to be learned are loaded into the mobile phone and played repeatedly in the user's earpiece while s/he performs other tasks. As each note of the song plays, a vibrator on the appropriate finger in the gloves activates, indicating which finger is used to play the note. MMT users showed significant improvement in their ability to reproduce simple piano melodies, even though they were actively engaged in a reading comprehension task during practice. The PI terms this effect Passive Tactile Learning (PTL). He also has initial evidence for a related effect, Passive Tactile Rehabilitation (PTR), where tactile stimulation of the hand correlates to improvements in dexterity and sensation for people with tetraplegia resulting from incomplete spinal cord injury. In this project, the PI will explore PTL/PTR effects in a variety of applications, develop procedural and device design guidelines to best elicit these effects, and illustrate methods and metrics that are sensitive to detecting these effects.Using MMT, he will compare users' learning of piano melodies while being distracted by a math-intensive test in four conditions: vibration stimulation with accompanying audio, vibration alone, audio alone, and no stimulation. This study will quantify the PTL effect and establish if vibration alone is sufficient to cause a learning effect. In an effort to broaden the applicability of the method, he will study the use of PTL to aid in learning stenographic typing and dance steps. To explore the possibility of PTR, he will run a 12-18 person controlled study with participants with tetraplegia due to partial spinal cord injury. This "in-the-wild" PTL/PTR study will compare active practice on a piano with active practice augmented with MMT. Participants will wear the glove during their everyday lives for at least 2 hours a day. The PI will compare learning rates as well as changes in participants' scores on standard sensation and dexterity tests. And he will also conduct a wearability study on MMT for persons with spinal cord injury to better accommodate their needs.Broader Impacts: Establishing guidelines for the use and effectiveness of PTL and PTR will enable others to apply the concepts in different domains, e.g., learning sign language or manual procedures such as a pre-flight checklist or training users of prosthetic limbs in how to trigger different actions in their limb. Beyond partial spinal cord injury, PTR might also be exploited for recovery from damage from stroke or lesions due to Multiple Sclerosis.The PI maintains an aggressive outreach program. Beyond professional publication of results in both the Human Computer Interaction and Rehabilitation literature, he will leverage his contacts to present the work in mainstream press and media (e.g., CNN), as well as more niche communities such as the Georgia Radio Relay Service. One of his focuses is exposing children in Georgia area schools for the deaf to his research, as there is a distinct need for encouraging deaf children in STEM areas. This effort will also be part of the undergraduate "Device Thread" in Georgia Tech's College of Computing, which exposes undergraduates to the rigors of designing computing for situations with physical device limitations, such as mobile phones, wearable devices, and robotics. Graduates and undergraduates in these classes will help investigate and fabricate new PTL/PTR devices. Finally, the PI intends to use this effort in his Computing4Good campaign, which inspires students to think of computing as a means for enabling social change.