A multimodal interface for real-time soldier-robot teaming
A multimodal interface for real-time soldier-robot teaming
复制标题
用于实时士兵机器人组队的多模式界面
DOI：
10.1117/12.2224401
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Matthew R. Walter
中科院分区：
文献类型：
--
作者：
D. Barber;T. Howard;Matthew R. Walter
机器人技术的最新研究和进展促进了利用新传感功能进行语义导航的新型平台的开发。随着这些系统变得越来越强大，它们支持除直接远程操作和路径点查找之外的高度复杂的命令，从而促进从机器人作为工具到机器人作为队友的转变。支持未来的士兵-机器人团队需要与人类团队同等的通信能力，才能成功集成机器人。因此，随着机器人功能的增加，士兵和机器人之间的接口也同样重要。多模式通信 (MMC) 通过冗余和比单模式交互更强大的通信级别实现人机协作。近年来针对智能手机和游戏发布的商用现成 (COTS) 技术提供了通过使用语音、手势和视觉显示来创建结合 MMC 的便携式界面的工具。然而，为了使多模态接口成功应用于军事领域，它们必须能够高精度地实时分类语音、手势和处理自然语言。在本研究中，开发了一个支持与自主机器人实时交互的原型多模式界面。该设备在平板电脑上集成了 COTS 自动语音识别 (ASR)、定制手势识别手套和自然语言理解功能。本文介绍了集成设备在命令自主机器人在未知的室外环境中执行侦察和监视活动时的性能结果（例如响应时间、准确性）。
Recent research and advances in robotics have led to the development of novel platforms leveraging new sensing capabilities for semantic navigation. As these systems becoming increasingly more robust, they support highly complex commands beyond direct teleoperation and waypoint finding facilitating a transition away from robots as tools to robots as teammates. Supporting future Soldier-Robot teaming requires communication capabilities on par with human-human teams for successful integration of robots. Therefore, as robots increase in functionality, it is equally important that the interface between the Soldier and robot advances as well. Multimodal communication (MMC) enables human-robot teaming through redundancy and levels of communications more robust than single mode interaction. Commercial-off-the-shelf (COTS) technologies released in recent years for smart-phones and gaming provide tools for the creation of portable interfaces incorporating MMC through the use of speech, gestures, and visual displays. However, for multimodal interfaces to be successfully used in the military domain, they must be able to classify speech, gestures, and process natural language in real-time with high accuracy. For the present study, a prototype multimodal interface supporting real-time interactions with an autonomous robot was developed. This device integrated COTS Automated Speech Recognition (ASR), a custom gesture recognition glove, and natural language understanding on a tablet. This paper presents performance results (e.g. response times, accuracy) of the integrated device when commanding an autonomous robot to perform reconnaissance and surveillance activities in an unknown outdoor environment.
虚拟、增强和混合现实设计和开发虚拟和增强环境
DOI：
10.1007/978-3-319-07458-0
发表时间：
2014
期刊：
影响因子：
--
作者：
Stelzer;Steindecker
通讯作者：
Steindecker