Multimicrophone Adaptive Array Augmented with Visual Cueing

描述（由申请人提供）：Advanced Medical Electronics Corporation（AME）提出开发一种基于智能手机的视听阵列，该视听阵列使得助听器用户能够在具有显著的语音混串音噪声的混响环境中与多个扬声器匡威，在混响环境中，他们的助听器不能很好地工作。该阵列将由智能手机、智能手机配件和智能手机软件应用程序组成。智能手机多麦克风视听阵列附件将采用允许附接到智能手机背面的形状因子。该附件还将包含一个低功率无线电，通过该无线电可以将音频信号传输到兼容的助听器。智能手机软件应用程序将使用智能手机的内置摄像头来获取图像，并使用智能手机的内置面部检测支持来执行实时面部检测。会说话的目标的位置将被用于 以实现视听波束形成算法来提高信噪比，从而提高用户的语音清晰度。由于所提出的阵列系统利用手持消费电子设备，因此它将是便携式的且成本低。AME将在第二阶段设计生产原型并进行验证。

DESCRIPTION (provided by applicant): Advanced Medical Electronics Corporation (AME) proposes to develop a smartphone-based audiovisual array that enables hearing aid users to converse with multiple speakers in reverberant environments with significant speech babble noise where their hearing aids do not function well. The array will consist of a smartphone, a smartphone accessory, and a smartphone software application. The smartphone multi-microphone audiovisual array accessory will be in a form factor that allows attachment to the back of the smartphone. The accessory will also contain a lower power radio by which it can transmit audio signals to compatible hearing aids. The smartphone software application will use the smartphone's built in camera to acquire images and perform real-time face detection using the built-in face detection support of the smartphone. The location of talking targets will be used to implement an audiovisual beamforming algorithm to improve the signal to noise ratio and consequently improve the user's speech intelligibility. Since the proposed array system leverages a handheld consumer electronic device, it will be portable and low cost. AME will design a production prototype in phase II and perform validation.