HCC: Small: Passive Tactile Learning and Rehabilitation Using Wearable Computers

移动的音乐触摸（MMT），由PI和他的同事在先前的工作中开发的，是一个轻量级的，无线触觉音乐教学系统，由无指手套和蓝牙功能的移动的手机组成。 要学习的钢琴段落被加载到移动的电话中，并在用户的耳机中反复播放，同时她/他执行其他任务。 当歌曲的每个音符播放时，手套中相应手指上的振动器会激活，指示哪个手指用于播放音符。 MMT使用者表现出显着的改善，他们的能力，重现简单的钢琴旋律，即使他们积极从事阅读理解任务在实践中。 PI将这种效应称为被动触觉学习（PTL）。 他还初步证明了一种相关的效果，被动触觉康复（PTR），其中手部的触觉刺激与不完全脊髓损伤所致四肢瘫痪患者的灵巧性和感觉改善相关。 在这个项目中，PI将探索各种应用中的PTL/PTR效应，开发程序和设备设计指南以最好地引出这些效应，并说明对检测这些效应敏感的方法和指标。使用MMT，他将比较用户在四种情况下学习钢琴旋律时被数学密集型测试分心：伴随音频的振动刺激、单独振动、单独音频和无刺激。 本研究将量化PTL效应，并确定单独振动是否足以引起学习效应。 为了扩大该方法的适用性，他将研究使用PTL来帮助学习速记打字和舞步。 为了探索PTR的可能性，他将对因部分脊髓损伤而四肢瘫痪的参与者进行一项12-18人对照研究。 这项“野外”PTL/PTR研究将比较钢琴上的主动练习与MMT增强的主动练习。 参与者将在日常生活中佩戴手套，每天至少2小时。 PI将比较学习率以及参与者在标准感觉和灵活性测试中得分的变化。 更广泛的影响：为使用PTL和PTR及其有效性制定指引将使其他人能够在不同领域应用这些概念，例如，学习手语或手动程序，如飞行前检查清单或培训假肢使用者如何触发肢体的不同动作。 除了部分脊髓损伤外，PTR还可用于中风或多发性硬化症损伤的恢复。PI保持积极的外展计划。 除了在人机交互和康复文献中专业出版结果外，他还将利用他的联系人在主流新闻和媒体上展示工作（例如，CNN），以及更多的利基社区，如格鲁吉亚无线电中继服务。 他的重点之一是让格鲁吉亚地区聋哑学校的孩子们了解他的研究，因为在STEM领域鼓励聋哑儿童是非常必要的。 这项工作也将成为格鲁吉亚理工学院计算学院本科生“设备线程”的一部分，该课程使本科生接触到为物理设备限制（如移动的手机、可穿戴设备和机器人技术）的情况设计计算的严格要求。 这些课程的毕业生和本科生将帮助研究和制造新的PTL/PTR设备。 最后，PI打算在他的Computing 4Good活动中使用这一努力，该活动激励学生将计算视为实现社会变革的一种手段。

Mobile Music Touch (MMT), developed in prior work by the PI and his colleagues, is a lightweight, wireless tactile music instruction system consisting of fingerless gloves and a Bluetooth-enabled mobile phone. Piano passages to be learned are loaded into the mobile phone and played repeatedly in the user's earpiece while s/he performs other tasks. As each note of the song plays, a vibrator on the appropriate finger in the gloves activates, indicating which finger is used to play the note. MMT users showed significant improvement in their ability to reproduce simple piano melodies, even though they were actively engaged in a reading comprehension task during practice. The PI terms this effect Passive Tactile Learning (PTL). He also has initial evidence for a related effect, Passive Tactile Rehabilitation (PTR), where tactile stimulation of the hand correlates to improvements in dexterity and sensation for people with tetraplegia resulting from incomplete spinal cord injury. In this project, the PI will explore PTL/PTR effects in a variety of applications, develop procedural and device design guidelines to best elicit these effects, and illustrate methods and metrics that are sensitive to detecting these effects.Using MMT, he will compare users' learning of piano melodies while being distracted by a math-intensive test in four conditions: vibration stimulation with accompanying audio, vibration alone, audio alone, and no stimulation. This study will quantify the PTL effect and establish if vibration alone is sufficient to cause a learning effect. In an effort to broaden the applicability of the method, he will study the use of PTL to aid in learning stenographic typing and dance steps. To explore the possibility of PTR, he will run a 12-18 person controlled study with participants with tetraplegia due to partial spinal cord injury. This "in-the-wild" PTL/PTR study will compare active practice on a piano with active practice augmented with MMT. Participants will wear the glove during their everyday lives for at least 2 hours a day. The PI will compare learning rates as well as changes in participants' scores on standard sensation and dexterity tests. And he will also conduct a wearability study on MMT for persons with spinal cord injury to better accommodate their needs.Broader Impacts: Establishing guidelines for the use and effectiveness of PTL and PTR will enable others to apply the concepts in different domains, e.g., learning sign language or manual procedures such as a pre-flight checklist or training users of prosthetic limbs in how to trigger different actions in their limb. Beyond partial spinal cord injury, PTR might also be exploited for recovery from damage from stroke or lesions due to Multiple Sclerosis.The PI maintains an aggressive outreach program. Beyond professional publication of results in both the Human Computer Interaction and Rehabilitation literature, he will leverage his contacts to present the work in mainstream press and media (e.g., CNN), as well as more niche communities such as the Georgia Radio Relay Service. One of his focuses is exposing children in Georgia area schools for the deaf to his research, as there is a distinct need for encouraging deaf children in STEM areas. This effort will also be part of the undergraduate "Device Thread" in Georgia Tech's College of Computing, which exposes undergraduates to the rigors of designing computing for situations with physical device limitations, such as mobile phones, wearable devices, and robotics. Graduates and undergraduates in these classes will help investigate and fabricate new PTL/PTR devices. Finally, the PI intends to use this effort in his Computing4Good campaign, which inspires students to think of computing as a means for enabling social change.