An Active Audition Framework for Auditory-driven HRI: Application to Interactive Robot Dancing
An Active Audition Framework for Auditory-driven HRI: Application to Interactive Robot Dancing
复制标题
听觉驱动 HRI 的主动试听框架：在交互式机器人舞蹈中的应用
DOI：
10.1109/roman.2012.6343892
复制
发表时间：
2012
期刊：
影响因子：
--
通讯作者：
Fabien Gouyon
中科院分区：
文献类型：
--
作者：
Joao Lobato Oliveira;Gokhan Ince;Keisuke Nakamura;Kazuhiro Nakadai;Hiroshi G. Okuno;Luis Paulo Reis;Fabien Gouyon
在本文中，我们提出了一个通用的主动听觉框架的语义驱动的人机交互（HRI）。该框架同时处理语音和音乐的飞行，集成感知模型的机器人试听，并支持口头和非口头的交互式交流（亲）主动行为。为了保证交互的可靠性，在框架的基础上，提出了基于主动听觉策略的行为决策机制，根据听觉信号的可靠性对机器人的动作进行决策。为了验证该框架的应用程序一般知识驱动的HRI，我们提出了一个交互式机器人跳舞系统的实现。该系统集成了三个预处理机器人听觉模块：声源定位，声源分离和自我噪声抑制;两个听觉感知模块：现场音频节拍跟踪和自动语音识别;和多模态行为的口头和非口头的互动：音乐驱动的舞蹈和语音驱动的对话。为了全面评估该系统，我们设置了具有高度动态声学条件的实验和交互式真实场景，并定义了一套评估标准。实验测试显示了准确和强大的节拍跟踪和语音识别，以及令人信服的舞蹈节拍同步。交互式会议确认了行为决策机制在积极维护鲁棒和自然的人机交互方面的基本作用。
In this paper we propose a general active audition framework for auditory-driven Human-Robot Interaction (HRI). The proposed framework simultaneously processes speech and music on-the-fly, integrates perceptual models for robot audition, and supports verbal and non-verbal interactive communication by means of (pro)active behaviors. To ensure a reliable interaction, on top of the framework a behavior decision mechanism based on active audition policies the robot's actions according to the reliability of the acoustic signals for auditory processing. To validate the framework's application to general auditory-driven HRI, we propose the implementation of an interactive robot dancing system. This system integrates three preprocessing robot audition modules: sound source localization, sound source separation, and ego noise suppression; two modules for auditory perception: live audio beat tracking and automatic speech recognition; and multi-modal behaviors for verbal and non-verbal interaction: music-driven dancing and speech-driven dialoguing. To fully assess the system, we set up experimental and interactive real-world scenarios with highly dynamic acoustic conditions, and defined a set of evaluation criteria. The experimental tests revealed accurate and robust beat tracking and speech recognition, and convincing dance beat-synchrony. The interactive sessions confirmed the fundamental role of the behavior decision mechanism for actively maintaining a robust and natural human-robot interaction.