Study on Computational Auditory Scene Analysis for Humanoids by Active Audition

机器人试听是一种能让人形机器人通过安装在身体上的麦克风（耳朵）听到声音的能力。由于在现实世界中，类人机器人通常会听到混合的声音，因此需要计算听觉场景分析（CASA）来实现同时听几件事情的能力，例如“Shotoku- taishi”（太子正德）。CASA的基本功能是声源定位、分离和识别分离的声音。我们获得了以下研究成果：1)较少先验信息的CASA功能：基于缺失特征的方法通过开发自动缺失特征掩模生成，集成了声音定位（MUSIC，或导向波束形成器）、声源分离（几何源分离，或独立分量分析）和自动语音识别（多频带Julius，或CTK）。整个系统在FlowDesigner架构上实现，实现了三种同时语音的识别，延迟时间为1.9秒，验证了该方法在SIG2、roboview - r2和asimo等不同类人机器人上的有效性。2)基于距离的行为选择：根据Proxemics设计基于类人机器人与人之间距离的交互策略，选择合适的交互伙伴。该系统在SIG-2人形机器人上实现，并在京都大学博物馆进行了为期三个月的演示，以验证其在多人交互中的有效性。3)基于基于最近邻分类器的颜色目标检测，开发了鲁棒人脸跟踪，提高了移动说话人跟踪的性能。4)开发了多声部音乐信息技术，包括乐器识别、鼓声提取、歌手识别等。5)为提高多域语音对话系统的可用性，提出了用户模型和语音识别错误恢复方法。6)开发了将环境声音用于人机交互的拟声词自动识别系统。未来的工作包括基于CASA的机器人试听系统的设计和开发。少

Robot audition is a capability in which a humanoid can hear sounds with its own microphones (ears) mounted on its body. Since humanoids usually hear a mixture of sounds in the real world, Computational Auditory Scene Analysis (CASA) of which essential functions consist of sound source localization, separation, and recognition of separated sounds is required to realize the capability of listening to several things simultaneously, like "Shotoku-Taishi" (Prince Shotoku). We have obtained the following research results :1) CASA functions with less prior information :The missing-feature based approach integrated sound localization (MUSIC, or steered beamformer), sound source separation (Geometrical Source Separation, or Independent Component Analysis), and automatic speech recognition (Mulit-band Julius, or CTK) by developing automatic missing feature mask generation. The whole system was implemented on the FlowDesigner architecture, so that recognizing three simultaneous speech was perform … More ed with latency of 1.9 sec. This result confirmed the validity of our approach on different humanoids including SIG2, Robovie-R2, and ASIMO.2) Distance-based behavior selection :The interaction strategy based on the distance between the humanoid and people according to Proxemics was devised to select an appropriate interaction partner. This system implemented on SIG-2 Humanoid was demonstrated for three months at the Kyoto University Museum to confirm its effectiveness in multiple person interaction.3) Robust face tracking was developed based on Color-target Detection Based on Nearest Neighbor Classifier to improve the performance of moving talker tracking.4) Music information technologies for polyphonic music, including musical instrument recognition, drum sound extraction, and singer recognition, were developed for humanoids to hear music.5) User model and error recovery from speech recognition errors were developed to improve the usability of multi-domain spoken dialogue system.6) Automatic onomatopoeia recognition system was developed to use environmental sounds in humanoid-human interaction.Future work includes the design and development of robot audition based on CASA. Less