Deep Animal Linguistic Analysis (DALA) – Decoding animal communication using a hybrid approach between bioacoustics and machine learning

今天我们对动物交流的理解非常有限。我们还远远不能确定动物声音的详细模式及其可能的含义。为了确定具有统计意义的观察结果，有必要分析大量的动物声学和行为数据。为了获得足够的观察数据，越来越多的视听媒体被用于研究各种动物的交流和行为。由于观察者的悖论，记录必须以一种不引人注目的方式进行，以尽可能少地分散动物的注意力。因此，这些数据集包含大量不相关的信号，例如环境噪声，只有少量的动物相互作用。强大的自动化机器方法能够分析如此大的数据集合。一个主要的障碍是目前生物声学学家可用的分析软件的技术限制，阻碍了动物交流研究的快速进展。目前可用的生物声学分析工具仅提供基本功能，如波形显示，频谱图，简单的音频处理选项和一些注释功能。深层动物语言分析（DALA）在生物声学和机器学习研究领域之间架起了一座桥梁，旨在开发新一代开源分析工具，能够自动处理大量复杂的动物发声。DALA使用最新的深度学习和其他模式识别技术，以便能够(1)在大型和嘈杂的数据集中分割和分离动物的发声，(2)自动识别有意义和不同发声类别的清单，(3)生成组合语言（语义和句法）模式，以及(4)建立动物特定的语言模型。通过这种方式，我们将促进衍生语言模型与相关情景视频记录和行为描述的系统交叉比较，以隔离重复出现的匹配，这再次允许检测有关潜在有意义的通信和行为相关性的信息。通过填补这种技术和方法上的模式识别空白，由此产生的方法/工具具有将生物/生物声学研究领域加速到一个新水平的现实潜力。在整个项目中，我们将与认知科学研究所、osnabr<e:1>和莱布尼茨动物园和野生动物研究所密切合作。该项目坚持开放科学原则（开放数据、开放源代码和开放获取）。

Today’s understanding of animal communication is very limited. We are still far away from identifying detailed patterns of animal sounds and their possible meanings. In order to determine statistically significant observations, it is necessary to analyze large amounts of animal acoustic and behavioral data. To generate sufficient observation data, audiovisual media are increasingly used to study communication and behavior of various kinds of animals. Due to the observer’s paradox, recording has to be done in an unobtrusiveway to create as little distraction as possible for the animals. As a result, such datasets contain large numbers of irrelevant signals, e.g. environmental noise, with only a small amount of animal interactions.Robust automated machine approaches enable the analysis of such large data collections. A main obstacle are the technological limitations of today’s analysis software that is available to bioacousticians, hindering rapid progress in animal communication research. Currently available bioacoustic analysis tools provide only basic features, like the display of waveforms, spectrograms, simple audio processing options, and some annotation functions.Deep Animal Linguistic Analysis (DALA) forms a bridge between the bioacoustics and machine learning research field in order to develop a new generation of open source analysis tool, capable of automatically handling large amounts of complex animal vocalizations. DALA uses recent deep learning and other pattern recognition techniques in order to be able to (1) segment and separate animal vocalizations within large and noisy datasets, (2) automatically identify an inventory of meaningful and different vocalization categories, (3) generate combinatorial linguistic (semantic and syntactic) patterns, and (4) build an animal specific language model. In this way, we will facilitate a systematic cross-comparison of the derived language model against associated situational video recordings and behaviour descriptions to isolate reappearing matches, which again allow to detect information about potentially meaningful communication and behavioral correlations. By filling this technological and methodological pattern recognition gap, the resulting methods/tools have a realistic potential to accelerate the biological/bioacoustic research field to a new level. Over the entire project, we will be in close collaboration with the Institute of Cognitive Sciences, Osnabrück and Leibnitz Institute for Zoo and Wildlife Research. The project adheres to the open science principle (open data, open source, and open access).