CRII: CHS: TongueWrite: An efficient tongue-based text-entry method using Multifunctional intraORal Assistive technology (MORA)

高度瘫痪的患者可以通过增加或替换输入/控制模式来提高他们的独立性和生活质量。然而，目前许多辅助技术的输入模式是有限的。它们大多可以与特定的任务交互，例如计算机鼠标控制、轮椅驾驶或文本输入，但不能同时用于多种目的。即使这些技术与多种模态接口，它们也不是非常用户友好，残疾人可能会选择不使用效率低下的模态，或者从一种技术切换到另一种技术来完成不同的任务。人体与辅助技术互动最有力的候选者之一是舌头。人类的舌头能够控制严重残疾的人脖子以上的自主运动。现有的基于舌头的辅助技术在光标导航和轮椅驾驶方面的舒适程度与使用手指相同，但它们在文本输入方面的表现并不好。本项目将探索基于语言的界面如何通过开发直观且易于学习的语言命令来提高文本输入的性能（除了导航和其他离散命令等任务）。最终，这种多模式将增强手部运动受限者（如四肢瘫痪、中风、帕金森病和与年龄相关的神经系统疾病）的独立性，以及他们的照顾者的生活质量。对于文本输入，用户应该能够明确地指定应被视为输入的键的位置和顺序。为了使用人类舌头进行文本输入，相关的算法应该能够处理连续的舌头运动（涉及多个自由度），识别与文本输入相关的运动，并检测和忽略“正常”的舌头运动，如吞咽唾液等。为了应对这些挑战，该项目将根据需要执行的命令数量对用户在基于舌头的命令上的表现进行建模，并研究有效的文本输入方法以及各种文本输入模式/机制（如触摸屏）。研究小组开发了一种新的基于舌头的辅助技术，多功能口腔内辅助技术（MORA），该技术利用了先进的传感器技术和智能数据融合算法，同时利用了舌头的力量。作为定制的无线耳机/固定器，MORA使用位于用户脸颊/上托盘附近的四个三轴磁传感器阵列。MORA可以区分用户自定义的舌头运动和其他自然的舌头运动，特别是那些没有任何示踪剂附着的说话和吞咽的舌头运动。为了使用MORA模拟舌头命令的可用性，研究小组将首先通过命令的数量来评估舌头的表现：5、7和9。为了通过命令数量及其学习效果来评估舌头的表现，研究团队将实施随机命令任务和基于Fitts定律的多向敲击任务。然后，团队将探索基于多笔画、触摸屏和手势识别三种机制的各种文本输入方法（如H4-Writer、OPTI II、Hex-O-Spell、Metropolis II、EdgeWrite、Multitap），以研究基于离散舌头命令的高效文本输入方法。MORA将通过医疗机构的焦点小组向用户介绍，如德克萨斯州布莱恩的德克萨斯脑和脊柱研究所，以及德克萨斯州大学奥斯汀分校的老年服务和长期护理卓越中心。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Individuals with high-level of paralysis can enhance their independence and quality-of-life by adding or replacing modes of input/control with the individuals' available voluntary motions. The input modes of many current assistive technologies, however, are limited. They can mostly interface with specific tasks such as computer mouse control, wheelchair driving, or text entry, but not for multiple of these purposes. Even if these technologies interfaces with multiple modalities, they are not very user-friendly, and a disabled individual may choose not to use the inefficient modality or switch from one technology to another for different tasks. One of the most powerful candidates in the human body to interact with assistive technologies is the tongue. The human tongue is able to harness voluntary movements above the neck in individuals with severe disabilities. Existing tongue-based assistive technologies perform cursor navigation and wheelchair driving with the same level of comfort as using a finger, but their performance in text entry is not good. This project will explore how the tongue-based interfaces can improve the performance of text entry (apart from tasks such as navigation and other discrete commands) by developing intuitive and easy-to-learn tongue commands. Ultimately, this multimodality will enhance both the independence of those with limited hand motions (e.g., Tetraplegia, stroke, Parkinson's disease, and age-related neurological disorders) and the quality-of-life of their caregivers.For text entry, a user should be able to unambiguously specify the position and sequence of keys that should be considered as input. In order to use the human tongue for text entry, the associated algorithm should be able to handle the continuous tongue motion (involving multiple degrees of freedom), identify the movements related to text entry, and detect and ignore “normal” tongue movements such as swallowing saliva, etc. To address these challenges, this project will model the performance of users on tongue-based commands by the number of commands to be carried out and investigate efficient text entry methods along with various modes/mechanisms of text entry (such as a touchscreen). The research team has developed a new tongue-based assistive technology, Multifunctional intraORal Assistive technology (MORA), which employs advanced sensor technology and a smart data fusion algorithm while taking advantage of the power of the tongue. Designed as a customized wireless headset/retainer, MORA uses an array of four three-axis magnetic sensors located near user’s cheek / upper pallet. MORA can differentiate user-defined tongue movements from other natural tongue movements, especially those involved in speaking and swallowing without any tracer attachment. To model the usability of the tongue commands using MORA, the research team will first evaluate tongue performance by the number of commands: five, seven, and nine. To evaluate tongue performance by the number of commands and their learning effects, the research team will implement a random command task and a Fitts' law-based multidirectional tapping task. Then, the team will explore various text entry methods (such as H4-Writer, OPTI II, Hex-O-Spell, Metropolis II, EdgeWrite, Multitap) based on three mechanisms: multi-stroke, touchscreen, and gesture recognition, to investigate efficient text entry methods based on discrete tongue commands. MORA will be introduced to users through focus groups in medical facilities such as The Texas Brain and Spine Institute, Bryan, Texas, and the Center for Excellence in Aging Services and Long-Term Care, in the University of Texas at Austin.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.