Unsupervised audio-visual geometry calibration of distributed microphone arrays

该项目的总体目标是简化视听传感器网络的安装。这是通过开发用于根据混响语音输入自动确定分布式麦克风阵列的位置的算法来实现的。然后，建立由声传感器网络感测的事件与由另一通道的传感器网络感测的事件之间的对应关系。这种另一种形式由多摄像机网络给出，其中假定摄像机位置是已知的。在第一个项目阶段，我们开发了二维麦克风阵列位置自校准算法，该算法的特点是不受混响影响，可以在语音输入上操作，不需要人工校准信号。所请求的项目扩展致力于以下目标：1)建立声传感器网络的相对几何和全球坐标系之间的对应关系，该坐标系由多个摄像机系统给出。这将通过将从声学子系统获得的说话人的轨迹映射到视觉相关者的轨迹，即人脸或人物来实现。2)开发的几何自校准系统将被嵌入环境通信系统中用于评估目的。在环境通信系统中，声学波束形成器提取语音信号，而活动摄像系统跟踪说话人。在该试验台中，可以在实时环境和现实环境条件下研究用户与系统的交互，最终实现校准系统的优化。

The overall goal of the project is to simplify the installation of audio-visual sensor networks. This is achieved by developing algorithms for automatically determining the position of the distributed microphone arrays from reverberant speech input. Then correspondences between events sensed by the acoustic sensor network and those sensed by a sensor network of another modality are established. This other modality is given by a multi-camera-network, where the camera positions are assumed to be known. Once the correspondences have been detected the relative geometries of the acoustic sensor network can be mapped to the given coordinate system of the camera network.In the first project phase we have developed algorithms for microphone array posiltion self calibration in 2D, which are characterized by immunity to reverberation and which can operate on speech input, not requiring artificial calibration signals.The requested project extension is devoted to the following objectives:1) A correspondence will be developed between the relative geometries of the acoustic sensor network and a global coordinate system, which is given by a multi-camera system. This shall be achieved by mapping trajectories of speakers obtained from the acoustic subsystem to trajectories of the visual correlates, i.e., faces or persons.2) The developed geometry self calibration system will be embedded in an ambient communication system for evaluation purposes.In the ambient communication system an acoustic beamformer extracts the speech signal, while the active camera system tracks the speaker. In this test bed the interaction between user and system can be studied in a real-time environment and under realistic environmental conditions, eventually leading to an optimization of the calibration system.