Text Entry by Inference: Eye Typing, Stenography, and Understanding Context of Use

我的研究是基于这样一种观察，即我们与计算机的日常互动是高度多余的。其中一些冗余可以通过智能用户界面进行建模和利用。智能文本输入方法使用机器学习等人工智能技术来利用我们语言中的冗余。它们使用户能够快速准确地书写，而不需要为每个想要的字母按键。在这个节目中，我建议开发两种新的智能文本输入方法。第一个是一个系统，它使残疾用户能够使用眼球跟踪器高效地进行交流。第二个系统是一种受速记启发的新型智能文本输入方法。此外，我还建议探索文本输入方法的更广泛的背景。研究文献集中在发明承诺高输入率和低错误率的文本输入方法。既然我们已经有了具有相当高的输入率的文本输入方法，是时候通过发现文本输入的其他方面来补充这个目标函数了。我建议使用社会科学技术，如日记和实地研究，来了解用户在野外更喜欢使用文本输入方法。系统1：通过推理进行眼部打字这是一个潜在地提高眼部打字系统入门率的系统。目前的眼睛打字系统天生就很慢(由于驻留超时)，用户认为它们令人沮丧。我建议建立一个系统，使用户能够在根本不需要停留超时的情况下进行眼部打字。潜在地，我的方法将比世界上任何其他基于眼球跟踪器的方法都要快。在我提出的系统中，用户通过将目光按顺序指向预期的字母键来书写单词。当用户看到位于键盘上方的结果区域时，他们想要的单词就会被转录。用户可以写一个以上的单词。他们还可以写出序列或单词，甚至可以在单词中稍作停顿。他们可能会进入单词之间的空格键，但这并不是系统能够正确推断用户想要的单词的严格要求。系统2：通过推理进行速记该系统将是一种用于笔或单手指输入的速记系统。主要应用是移动文本输入。然而，我努力创造一种系统，在某种程度上可以取代桌面键盘，如果用户愿意的话。潜在地，它将比任何其他基于笔的文本输入方法更快。这种方法背后的想法是使用户能够通过手势输入他们之前学习的模式来快速书写单词。这种来自肌肉记忆的开环回忆比用户在点击屏幕键盘时需要进行的闭环式视觉引导动作要快得多。我提出的系统将使用户能够快速准确地表达每个单词的手势。这些手势将针对特定的单词进行固定。也就是说，每个单词都与单一的(典型的)独特手势模式相关联。用户的输入手势由模式识别器识别。最接近模式与用户输入手势最匹配的单词将由系统输出为用户的意向单词。理解文本输入的更广泛上下文我提出的计划的最后一个组成部分将为文本输入研究领域提供新的视角。如前所述，在文本输入中很大程度上没有探索使用的上下文。我打算用一系列定性的方法来探讨这个话题。我打算进行访谈，进行实地研究(例如，研究参与者在咖啡馆尝试移动语音识别器的原型)，以及日记研究。后者将通过一个系统进行，该系统向用户提供几种文本输入方法的选择，我假设这些方法将适用于不同的情况。我还打算阅读有关设计和建筑的文献，以加深我对文本输入的完整设计空间的理解。

My research is based on the observation that our daily interaction with computers is highly redundant. Some of these redundancies can be modelled and exploited by intelligent user interfaces. Intelligent text entry methods use AI techniques such as machine learning to exploit redundancies in our languages. They enable users to write quickly and accurately, without the need for a key press for every single intended letter.In this programme I propose to develop two new intelligent text entry methods. The first is a system that enables disabled users to communicate efficiently using an eye-tracker. The second system is a novel intelligent text entry method that is inspired by stenography.In addition, I propose to explore text entry methods' broader context. The research literature has concentrated on inventing text entry methods that promise high entry rates and low error rates. Now that we have text entry methods that have reasonably high entry rates it is time to complement this objective function by discovering other aspects of text entry. I propose to use social-science techniques, such as diary and field-studies, to understand how users would prefer to use text entry methods in the wild. System 1: Eye-typing by inferenceThis is a system that will potentially increase the entry rate in eye-typing systems. Current eye-typing systems are inherently slow (due to the dwell timeouts), and users perceive them as frustrating. I propose to build a system that enables users to eye-type without the need for a dwell timeout at all. Potentially, my method will be faster than any other eye-tracker based method in the world.With my proposed system users write words by directing their gaze at the intended letter keys, in sequence. Users' intended words are transcribed when they look at a result area positioned above the keyboard. Users can write more than one word. They can also write sequences or words, or even stop short within a word. They may go to the spacebar key between words but this is not strictly necessary for the system to be able to correctly infer users' intended words.System 2: Stenography by inferenceThis system will be a stenography system for pen or single-finger input. The primary application is mobile text entry. However, I strive to create a system that to some extent can replace the desktop keyboard, should users so desire. Potentially it will be faster than any other pen-based text entry method.The idea behind this method is to enable users to write words quickly by gesturing patterns they have previously learned. Such open-loop recall from muscle-memory is much faster than the closed-loop visually-guided motions users are required to perform when they tap on, for example, an on-screen keyboard. My proposed system will enable users to quickly and accurately articulate gestures for individual words. These gestures will be fixed for a particular word. That is, each word is associated with a single (prototypical) unique gestural pattern. A user's input gesture is recognised by a pattern recognizer. The word whose closest pattern best match the user's input gesture will be outputted by the system as the user's intended word.Understanding the broader context of text entryThe last component of my proposed programme serves to contribute new perspectives to the text entry research field. As previously discussed, context of use is largely unexplored in text entry. I intend to explore this topic using a range of qualitative methods. I intend to perform interviews, conduct field studies (e.g. studying participants trying a prototype mobile speech recognizer at a caf), and diary-studies. The latter will be conducted with a system that provides users of a choice of a few text entry methods that I hypothesize will be useful for different situations. I also intend to read literature on design and architecture to further my understanding of the complete design space of text entry.