Sound recognition with limited supervision over sensor networks

机器学习系统的一个普遍问题是训练数据和测试数据之间的不匹配，导致性能显著下降。对于声学传感器网络(ASN)中的声音识别来说，由于声音和声学环境的巨大数量和变化性，以及由于人们可能遇到的各种传感器位置和几何结构，使得问题变得更加严重。因此，现有的声音识别数据库几乎永远不会完美地适合声传感器网络中的任何具体目标应用。本项目的主要目标是设计技术，以利用现有资源为ASN中的特定目标应用开发高性能的声事件和场景分类器。一方面，那些可用的资源是弱标签数据(仅用Event类标注的数据，而不是用时间开/偏移信息标注的数据)，这些数据来自与要为其开发应用的目标域不同的域。另一方面，我们假设来自目标域的大量未标记的音频记录是可用的。我们将开发技术来计算强标签(事件类别加上开启/偏移时间)，计算领域不变特征，并进行领域适配。我们还将考虑在测试期间进行适应性调整，以适应动态环境和传感器配置。应用的主要方法将是深度生成模型。我们将开发神经模型和适当的目标函数来分离变异的来源，特别是将与音频内容相关的变异与环境诱导的变异分开。此外，我们将开发检测异常声事件的方法，因为这些事件可能在应用程序中特别重要。

A general problem of machine learning systems is the mismatch between training and test data resulting in a significant degradation of performance. For sound recognition in acoustic sensor networks (ASNs) the problem is aggravated because of the huge number and variability of sounds and acoustic environments, and because of the large variety of sensor locations and geometric configurations one can encounter. Therefore, existing databases for sound recognition will almost never be a perfect fit to any concrete target application in acoustic sensor networks.The main objective of this project is to devise techniques for making use of available resources for the development of high-performance acoustic event and scene classifiers for a specific target application in an ASN. Those available resources are on one hand weakly labeled data (data annotated only with the event class, but not with temporal on/offset information), which stem from a different domain than the target domain for which an application is to be developed. On the other hand we assume availability of lots of unlabeled audio recordings from the target domain. We will develop techniques to compute strong labels (event category plus on/offset times), to compute domain-invariant features, and to carry out domain adaptation. We will also consider adaptation during test to account for dynamic environments and sensor configurations. The main methodology applied will be deep generative models. We will develop neural models and appropriate objective functions to disentangle sources of variation, in particular to separate audio content related variations from environment induced variations. Furthermore, we will develop methods to detect abnormal acoustic events, as those may be of particular interest in an application.