CAREER: Natural User Interfaces for Children

职业：儿童自然用户界面

• 批准号: 1552598
• 负责人: Lisa Anthony
• 金额: $ 49.36万
• 依托单位: University of Florida
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1552598&HistoricalAwards=false
• 关键词: CAREER; Natural; User; Interfaces; Children

Natural user interfaces allow users to interact with technology through modalities like touch, gesture, and motion. They are a key element in realizing the vision of ubiquitous computing, yet they present challenges with respect to supporting children. The PI's research on touchscreen interaction for children has found that existing surface gesture recognition algorithms designed, trained, and tested on adult input, and interaction design guidelines developed based on adult interaction patterns, do not apply equally well to children. For example, if a typical gesture interface is expecting a gesture to be entered as a single stroke, the system will not be able to process the multiple strokes generated by a child, leading to an unsuccessful interaction for that child. Recognition for whole-body interaction gestures experiences similar challenges; a child is more likely to perform an action or gesture (e.g., "jump" or "wave") with greater intensity or different motion paths than an adult performing the same gesture. In this research the PI's goal is to fundamentally advance our understanding of how to design and develop natural user interactions for children. The research will be carried out in three phases: (1) Data Collection and Analysis: collection of input behaviors from elementary-school aged children in each modality, and analysis for patterns and characteristics of children's input; (2) Recognition and Classification: development of new recognition algorithms attuned to the expected input behavior patterns of children and use of machine learning to evaluate their performance; and (3) Multimodal Interaction: investigation of multimodal input patterns exhibited by children, and validation of new approaches to multimodal synthesis that perform well on children's natural input. A testbed application will be developed to showcase the findings in an educational domain. Open-source natural user interaction recognition and synthesis algorithms and clear, practicable design recommendations will be developed and released in both peer-reviewed papers and on the project website for use by researchers and practitioners.This research will make fundamental advances in our understanding of child-computer interaction with natural user interfaces and develop robust new approaches for recognizing input and recovering from errors. This work will contribute solutions to interaction design research questions, such as how to best adapt and use these new modalities for children, and machine learning research questions, such as how to develop intelligent multimodal recognition algorithms tailored for children's input. The knowledge gained and contributions delivered by this research will inform the design of ubiquitous computing for children in learning contexts. To these ends, the PI will create models of expected input behavior patterns for children in two natural modalities, touchscreen interaction and whole-body interaction, and use these models to develop and adapt intelligent recognition algorithms tailored to process children's input. Multimodal interaction, or streamlined processing of disparate input from multiple simultaneous and unsynchronized input streams, is also a key component of natural user interfaces. Like unimodal recognition, traditional approaches to multimodal fusion for adult input may also not apply well to children. Specifically, the following research questions are targeted: What are the ways children produce interaction behaviors in multimodal natural user input modalities? What interaction design techniques are most effective for children using multimodal natural user input modalities? What new multimodal recognition and fusion algorithms for natural user interaction modalities perform effectively on children's input? This work will be conducted in partnership with schools and teachers, allowing the research findings to have immediate impact on how real children are using natural user interaction technology in educational contexts. The project will involve both undergraduate and graduate research assistants, broadening participation in computer science by recruiting women and underrepresented minority students. Education plans focus on developing a new undergraduate certificate in Human-Centered Computing.

自然用户界面允许用户通过触摸、手势和动作等方式与技术进行交互。 它们是实现无处不在的计算愿景的关键因素，但它们在支持儿童方面提出了挑战。 PI对儿童触摸屏交互的研究发现，现有的表面手势识别算法设计，训练和测试成人输入，以及基于成人交互模式开发的交互设计指南，并不同样适用于儿童。 例如，如果典型的手势界面期望将手势输入为单个笔划，则系统将不能处理由儿童生成的多个笔划，导致该儿童的不成功交互。 识别全身交互手势也面临类似的挑战;儿童更有可能执行动作或手势（例如，“跳”或“波”）具有比成人执行相同姿势更大的强度或不同的运动路径。 在这项研究中，PI的目标是从根本上提高我们对如何为儿童设计和开发自然用户交互的理解。 本研究将分三个阶段进行：（1）数据收集与分析：收集小学年龄段儿童在每种输入方式下的输入行为，分析儿童输入的模式和特征;（2）识别与分类：开发新的识别算法，以适应儿童预期的输入行为模式，并使用机器学习来评估他们的表现;（3）多模态交互：研究儿童表现出的多模态输入模式，并验证在儿童自然输入上表现良好的多模态合成新方法。 将开发一个测试平台应用程序，以展示在教育领域的研究结果。 开源的自然用户交互识别和合成算法以及清晰、实用的设计建议将在同行评议的论文和项目网站上发布，供研究人员和从业人员使用。这项研究将在我们对儿童与计算机自然用户界面交互的理解方面取得根本性进展，并开发出识别输入和从错误中恢复的强大新方法。 这项工作将有助于解决交互设计研究问题，例如如何最好地适应和使用这些新的儿童模式，以及机器学习研究问题，例如如何开发针对儿童输入的智能多模态识别算法。 本研究所获得的知识和贡献将为儿童学习环境中的普适计算设计提供信息。 为此，PI将以两种自然模式（触摸屏交互和全身交互）为儿童创建预期输入行为模式的模型，并使用这些模型开发和调整智能识别算法，以处理儿童的输入。 多模式交互，或对来自多个同时和非同步输入流的不同输入的流线化处理，也是自然用户界面的关键组件。 与单峰识别一样，传统的成人输入多模态融合方法也可能不适用于儿童。 具体而言，以下研究问题是有针对性的：孩子们产生交互行为的方式是什么？对于使用多模态自然用户输入方式的儿童来说，什么样的交互设计技术最有效？什么新的多模态识别和融合算法的自然用户交互方式有效地执行儿童的输入？ 这项工作将与学校和教师合作进行，使研究结果对真实的儿童如何在教育环境中使用自然用户交互技术产生直接影响。 该项目将涉及本科生和研究生研究助理，通过招募妇女和代表性不足的少数民族学生来扩大对计算机科学的参与。 教育计划的重点是开发一个新的以人为中心的计算本科证书。