Signal Capturing and Computing System for Interactive Applications in Communication Acoustics

提议的研究仪器将提供在交互式实验场景中捕获音频、视频和跟踪数据的方法，并使用信号处理和机器学习方法分析和建模这些数据。该系统旨在捕捉人们的自然对话，以及人们与技术设备（助听器、人工耳蜗、智能扬声器、智能手机、机器人）的互动。它将记录声学和视频数据以及这些人员和设备在声学实验室环境中的位置跟踪数据。这些数据将存储在研究设备中，并将使用集成的强大gpu服务器上训练的神经网络进行分析和建模。拟议的仪器将在两个重要方面支持科学方法：首先，它将提供有关声源和接收器的位置、方向和运动的地面真实信息的视听数据库的捕获。其次，它将被用于更深入地了解新型深度神经网络结构的特征和最优性，并与传统的分类和回归统计模型进行比较。由于该系统不是单一供应商提供的，它将由将从几个专业供应商购买的组件组装而成。该系统被配置为可用于大量正在进行和未来的研究项目，因此将适合用于通信声学研究的长期和密集使用。

The proposed research instrument will provide the means for capturing audio, video, and tracking data in interactive experimental scenarios and for analysing and modeling this data using signal processing and machine learning methodologies. The systems aims at capturing natural conversations of people and the interaction of people with technical devices (hearing aids, cochlear implants, smart speakers, smartphones, robots). It will record acoustic and video data as well as position tracking data of these persons and devices in an acoustic lab environment. This data will be stored in the research device and will be analyed and modelled using neural networks trained on an integrated powerful GPU-server.The proposed instrument will support the scientific methodology in two important aspects: firstly, it will provide capturing of audio-visual data bases with ground truth information concerning the position, orientation, and movements of acoustic sources and receivers. Secondly, it will be used to derive more thorough insights into the characteristics and optimality of novel deep neural network structures, also in comparison to traditional statistical models for classification and regression.As this system is not available from a single vendor it will be assembled from components which will be bought from several specialized suppliers. The system is configured to be useful in a large number of ongoing and future research projects and thus will be fit for a long and intensive use for research in communication acoustics.