CRII: CHS: TongueWrite: An efficient tongue-based text-entry method using Multifunctional intraORal Assistive technology (MORA)

CRII：CHS：TongueWrite：使用多功能口内辅助技术 (MORA) 的高效基于舌头的文本输入方法

• 批准号: 1948503
• 负责人: Kiju Lee
• 金额: $ 17.5万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1948503&HistoricalAwards=false
• 关键词: CRII; CHS; TongueWrite; efficient; tongue

Individuals with high-level of paralysis can enhance their independence and quality-of-life by adding or replacing modes of input/control with the individuals' available voluntary motions. The input modes of many current assistive technologies, however, are limited. They can mostly interface with specific tasks such as computer mouse control, wheelchair driving, or text entry, but not for multiple of these purposes. Even if these technologies interfaces with multiple modalities, they are not very user-friendly, and a disabled individual may choose not to use the inefficient modality or switch from one technology to another for different tasks. One of the most powerful candidates in the human body to interact with assistive technologies is the tongue. The human tongue is able to harness voluntary movements above the neck in individuals with severe disabilities. Existing tongue-based assistive technologies perform cursor navigation and wheelchair driving with the same level of comfort as using a finger, but their performance in text entry is not good. This project will explore how the tongue-based interfaces can improve the performance of text entry (apart from tasks such as navigation and other discrete commands) by developing intuitive and easy-to-learn tongue commands. Ultimately, this multimodality will enhance both the independence of those with limited hand motions (e.g., Tetraplegia, stroke, Parkinson's disease, and age-related neurological disorders) and the quality-of-life of their caregivers.For text entry, a user should be able to unambiguously specify the position and sequence of keys that should be considered as input. In order to use the human tongue for text entry, the associated algorithm should be able to handle the continuous tongue motion (involving multiple degrees of freedom), identify the movements related to text entry, and detect and ignore “normal” tongue movements such as swallowing saliva, etc. To address these challenges, this project will model the performance of users on tongue-based commands by the number of commands to be carried out and investigate efficient text entry methods along with various modes/mechanisms of text entry (such as a touchscreen). The research team has developed a new tongue-based assistive technology, Multifunctional intraORal Assistive technology (MORA), which employs advanced sensor technology and a smart data fusion algorithm while taking advantage of the power of the tongue. Designed as a customized wireless headset/retainer, MORA uses an array of four three-axis magnetic sensors located near user’s cheek / upper pallet. MORA can differentiate user-defined tongue movements from other natural tongue movements, especially those involved in speaking and swallowing without any tracer attachment. To model the usability of the tongue commands using MORA, the research team will first evaluate tongue performance by the number of commands: five, seven, and nine. To evaluate tongue performance by the number of commands and their learning effects, the research team will implement a random command task and a Fitts' law-based multidirectional tapping task. Then, the team will explore various text entry methods (such as H4-Writer, OPTI II, Hex-O-Spell, Metropolis II, EdgeWrite, Multitap) based on three mechanisms: multi-stroke, touchscreen, and gesture recognition, to investigate efficient text entry methods based on discrete tongue commands. MORA will be introduced to users through focus groups in medical facilities such as The Texas Brain and Spine Institute, Bryan, Texas, and the Center for Excellence in Aging Services and Long-Term Care, in the University of Texas at Austin.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高度瘫痪的人可以通过增加或用个人可用的自愿运动来增加或取代输入/控制模式来提高他们的独立性和生活质量。然而，目前许多辅助技术的输入模式都是有限的。它们大多可以与特定的任务交互，如计算机鼠标控制、轮椅驾驶或文本输入，但不能用于这些目的中的多个。即使这些技术与多个医疗设备接口，它们也不是非常用户友好的，残障人士可能选择不使用低效的医疗设备，或者为不同的任务从一种技术切换到另一种技术。在人体内与辅助技术互动的最强大的候选者之一是舌头。在严重残疾的人中，人类的舌头能够利用颈部以上的自发运动。现有的基于舌头的辅助技术执行光标导航和轮椅驾驶的舒适度与使用手指相同，但在文本输入方面的表现并不好。这个项目将探索基于舌头的界面如何通过开发直观和易于学习的舌头命令来提高文本输入的性能(除了导航和其他离散命令等任务)。最终，这种多模式将提高那些手部运动受限的人(例如四肢瘫痪、中风、帕金森病和与年龄相关的神经疾病)的独立性和他们照顾者的生活质量。对于文本输入，用户应该能够明确地指定应该被认为是输入的键的位置和顺序。为了使用人类的舌头进行文本输入，相关的算法应该能够处理舌头的连续运动(涉及多个自由度)，识别与文本输入相关的运动，并检测和忽略“正常”的舌头运动，如吞咽唾液等。为了解决这些挑战，该项目将根据要执行的命令的数量来模拟用户对基于舌头的命令的表现，并研究有效的文本输入方法以及各种文本输入模式/机制(如触摸屏)。该研究团队开发了一种基于舌头的新辅助技术-多功能口腔内辅助技术(Mora)，该技术采用了先进的传感器技术和智能数据融合算法，同时利用了舌头的力量。作为定制的无线耳机/固定器，Mora使用位于用户脸颊/上托盘附近的四个三轴磁性传感器阵列。Mora可以将用户定义的舌头运动与其他自然舌头运动区分开来，特别是那些涉及说话和吞咽而没有任何示踪物附着的舌头运动。为了模拟使用Mora的舌头命令的可用性，研究团队将首先通过命令的数量来评估舌头的性能：5、7和9。为了通过命令的数量及其学习效果来评估舌头的表现，研究团队将实施一项随机命令任务和一项基于Fitts定律的多方向敲击任务。然后，团队将探索基于多笔画、触摸屏和手势识别三种机制的各种文本输入方法(如H4-Writer、OPTI II、Hex-O-Spell、Metropolis II、EdgeWite、MultiTap)，以研究基于离散舌头命令的高效文本输入方法。Mora将通过医疗机构的焦点小组向用户介绍，如德克萨斯州布赖恩的德克萨斯大脑和脊柱研究所，以及德克萨斯大学奥斯汀分校的老龄化服务和长期护理卓越中心。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。