HCC: Small: Enabling and Exploring Natural Interaction

技术的进步使得在各种设备和环境中探索人机交互的新方法成为可能，以努力实现感觉更自然的交互。虽然从用户和商业角度来看都很有价值，但从更有原则的系统设计或认知科学角度来看，对新技术的使用并没有得到很好的理解。即使是建立一套设计原则的基本要素，也会产生更好的设计，并增加我们在高风险/高回报领域使用该技术的信心，例如第一响应者规划和控制。PI的长期目标是制定一套原则，规定如何设计既能实现自然的人机交互，又能通过理解人的因素而获得信息的系统。自然交互指的是人与人之间在认知上透明、毫不费力的多模式交流；这项工作旨在使这一点在人机交互中成为可能。由人为因素提供信息的设计考虑了对人类能力的理解(例如，注意力、对多个信息通道的使用等)，因此最终的系统与人类的信息处理是良好的阻抗匹配。这项研究将涉及建立按照这种精神设计的系统，并阐明其设计原则，这些原则反过来将通过明确一种或另一种模式适用的任务条件(例如，何时画图、何时交谈)以及在人机交流中有效地同时使用多种模式的方式来促进未来的设计。该项目是在一个基于桌面的系统的背景下进行的，该系统帮助城市搜索和救援(USAR)行动的指挥中心进行规划和协调。这项工作将通过利用和结合团队在构建新的交互技术和人为因素方面的经验来进行。他们将扩展PI的桌面系统的当前版本，该系统允许基本的笔式交互，从而使其能够处理现实世界USAR工作中使用的各种草图、手势和语音，从而提供更自然的交互风格。额外的交互方式将使系统更强大，而任务的真实世界、时间压力的特点为研究人的因素方面提供了一个很好的平台。这将使团队能够了解各种模式是如何、何时以及为什么有用的，并提供可以阐明系统设计原则的数据。为了帮助确保项目成果的适用性，PI将在第二个领域探索相同的问题，即使用UML图进行软件设计。智力优势：这项研究将通过产生关于通道选择的经验数据，并通过阐明一套广泛有用的界面设计原则来明确通道选择和认知有效通道组合的条件，从而深入了解多通道交互模型。这些原则为多模式界面设计提供了变革性改变的可能性，将其从目前主要是特别的实践改变为以可测试原则为指导的设计过程。在两个任务领域进行研究将有助于确保所得出的原则具有有用的普遍性。这项工作将在一定程度上改善急救队的效力，从而为社会带来好处。处理非传统交互模式(例如，手势)的能力最终将使身体条件较差的用户更容易访问计算机交互。私隐专员会尽可能使用和建立开放源码工具，以便向社会提供在项目过程中制作的所有研究软件，从而增加下一代研究和教育平台的供应。

Advances in technology are making it feasible to explore novel approaches to human-computer interaction in a wide variety of devices and settings, in an effort to achieve interaction that feels more natural. While valuable from both a user and commercial perspective, use of new technology is not well understood from a more principled system design or cognitive science perspective. Establishing even the basic elements of a set of design principles would both produce better designs and increase our confidence in using the technology in high-risk/high-reward domains, such as first responder planning and control. The PI's long-term goal is to develop a set of principles specifying how to design systems that both enable natural human-computer interaction and are informed by an understanding of human factors. Natural interaction refers to the cognitively transparent, effortless multimodal communication that can happen between people; this work aims to make that possible in human-computer interaction. Designs informed by human factors take into account an understanding of human capabilities (e.g., attention, use of multiple information channels, etc.), so that the final system is a good impedance match to human information processing. This research will involve building systems designed in this spirit and articulating principles for their design that, in turn, will facilitate future designs by making explicit both the task conditions under which one or another modality is appropriate (e.g., when to draw, when to talk), and the ways in which multiple modalities can effectively be used simultaneously in human-computer communication. The project is set in the context of a tabletop-based system that assists with planning and coordination in the command center of an urban search and rescue (USAR) operation. The work will proceed by leveraging and combining the team's experience in building novel interaction technologies and in human factors. They will extend the current version of the PI's tabletop system, which permits basic pen-based interaction, so as to give it the ability to handle the kinds of sketching, freehand gestures and speech used in real-world USAR work, thereby providing a far more natural style of interaction. The additional interaction modalities will make the system more powerful, while the real-world, time-pressured character of the task offers a good platform for studying the human factors aspects. This will allow the team to understand how, when and why various modalities are useful, providing the data from which system design principles can be articulated. To help ensure breadth of applicability of project outcomes, the PI will explore the same issues in a second domain, software design with UML diagrams.Intellectual Merit: This research will provide insight into a model of multimodal interaction by producing empirical data about modality selection, and by articulating a widely useful set of principles for interface design that make explicit the conditions for both modality selection and cognitively effective modality combination. Such principles offer the possibility of transformative change to multimodal interface design, changing it from the current largely ad hoc practice to a design process guided by testable principles. Pursuing the research in two task domains will help to ensure a useful degree of generality to the principles derived. The work will provide benefit to society to the extent it can improve the effectiveness of first responder teams. The ability to handle non-traditional interaction modalities (e.g., gesture) will ultimately make computer interaction more accessible to physically disadvantaged users. The PI will take care, to the extent possible, to use and build upon open source tools, so that he can make available to the community all of the research software produced during the course of the project, thereby adding to the supply of next-generation research and education platforms.