Designing Next-generation Mobile Interfaces for Dynamic Conversational Speech

交互式多模式界面：为人类性能设计这是PI持续奖的创意延伸。PI计划进行一项研究，包括对她的多模式系统进行移动测试。受试者将包括成年人和7至9岁的儿童。目标包括：(1)建立支持广泛的移动测试和半自动数据分析所需的完整研究基础设施；(2)记录多模式系统移动测试中的相互消歧，并研究与其增强相关的因素；(3)检查移动环境中系统识别性能与用户信号特征、环境噪声水平和信噪比信息之间的关系；以及(4)探索不同用户群中的移动语音信号模式和系统识别性能。这项研究将提供关于多模式界面设计的关键信息，这些设计适合于支持真实世界环境中和不同用户的稳健移动使用，特别是用于创建能够在逐个命令的基础上监控环境并智能地调整模式权重以避免识别故障和稳定系统性能的自适应多模式架构。在使用PI的移动系统期间收集的数据将有助于确定各种新的研究问题和界面设计挑战，这些问题和界面设计既没有被更广泛的研究社区认识到，也没有被探索

Interactive Multimodal Interfaces: Designing for Human PerformanceThis is a creativity extension to the PI's continuing award. The PI plans to conduct a study involving mobile testing of her multimodal system. Subjects will be a mixture of adults and 7-to-9-year-old children. The goals include: (1) establishing the full research infrastructure needed to support extensive mobile testing and semi-automated data analysis; (2) documenting mutual disambiguation during mobile testing of a multimodal system, and studying the factors associated with its enhancement; (3) examining the relation between system recognition performance in a mobile environment and users' signal characteristics, ambient noise levels, and signal-to-noise ratio information; and (4) exploring mobile speech signal patterns and system recognition performance in diverse user groups. The study will provide critical information about multimodal interface designs appropriate for supporting robust mobile use in real-world contexts and by varied users, and in particular for creating adaptive multimodal architectures that are capable of monitoring the environment on a command-by-command basis and adapting mode weights intelligently to avoid recognition failure and stabilize system performance. The data collected during the use of the PI's mobile system will assist in identifying a variety of new research issues and interface design challenges that have neither been recognized nor probed by the broader research community