Multimicrophone Adaptive Array Augmented with Visual Cueing

描述（由申请人提供）：Advanced Medical Electronics Corporation （AME）提议开发一种基于智能手机的视听阵列，使助听器用户能够在助听器功能不佳的混响环境中与多个扬声器进行交谈。该阵列将由智能手机、智能手机配件和智能手机软件应用程序组成。这款智能手机多麦克风视听阵列配件的外形可以安装在智能手机的背面。该配件还将包含一个低功率收音机，通过它可以将音频信号传输到兼容的助听器。智能手机软件应用程序将使用智能手机的内置摄像头获取图像，并使用智能手机的内置人脸检测支持进行实时人脸检测。将使用谈话目标的位置

DESCRIPTION (provided by applicant): Advanced Medical Electronics Corporation (AME) proposes to develop a smartphone-based audiovisual array that enables hearing aid users to converse with multiple speakers in reverberant environments with significant speech babble noise where their hearing aids do not function well. The array will consist of a smartphone, a smartphone accessory, and a smartphone software application. The smartphone multi-microphone audiovisual array accessory will be in a form factor that allows attachment to the back of the smartphone. The accessory will also contain a lower power radio by which it can transmit audio signals to compatible hearing aids. The smartphone software application will use the smartphone's built in camera to acquire images and perform real-time face detection using the built-in face detection support of the smartphone. The location of talking targets will be used to implement an audiovisual beamforming algorithm to improve the signal to noise ratio and consequently improve the user's speech intelligibility. Since the proposed array system leverages a handheld consumer electronic device, it will be portable and low cost. AME will design a production prototype in phase II and perform validation. PUBLIC HEALTH RELEVANCE: The primary complaint of hearing-aid users is their difficulty understanding speech in noise. Only 51% of hearing aid users report they are satisfied with their hearing aids in noise. A discrete device that augments hearing aids in difficult reverberant environments would allow hearing aid users to converse with multiple people in almost any environment including restaurants, automobiles, etc.