Signal Capturing and Computing System for Interactive Applications in Communication Acoustics

点击翻译按钮获取中文摘要

The proposed research instrument will provide the means for capturing audio, video, and tracking data in interactive experimental scenarios and for analysing and modeling this data using signal processing and machine learning methodologies. The systems aims at capturing natural conversations of people and the interaction of people with technical devices (hearing aids, cochlear implants, smart speakers, smartphones, robots). It will record acoustic and video data as well as position tracking data of these persons and devices in an acoustic lab environment. This data will be stored in the research device and will be analyed and modelled using neural networks trained on an integrated powerful GPU-server.The proposed instrument will support the scientific methodology in two important aspects: firstly, it will provide capturing of audio-visual data bases with ground truth information concerning the position, orientation, and movements of acoustic sources and receivers. Secondly, it will be used to derive more thorough insights into the characteristics and optimality of novel deep neural network structures, also in comparison to traditional statistical models for classification and regression.As this system is not available from a single vendor it will be assembled from components which will be bought from several specialized suppliers. The system is configured to be useful in a large number of ongoing and future research projects and thus will be fit for a long and intensive use for research in communication acoustics.