Automated annotation of birdsong with a neural network that segments spectrograms.
Automated annotation of birdsong with a neural network that segments spectrograms.
复制标题
DOI：
10.7554/elife.63853
复制
发表时间：
2022-01-20
期刊：
影响因子：
7.7
通讯作者：
Gardner TJ
中科院分区：
文献类型：
--
作者：
Cohen Y;Nicholson DA;Sanchioni A;Mallaber EK;Skidanova V;Gardner TJ
鸣禽为研究感觉-运动学习提供了一个强大的模型系统。然而，许多对鸟鸣的分析需要耗时的手动注释其元素，称为音节。已经提出了自动注释的方法，但这些方法假设音频可以被清晰地分割成音节，或者它们需要仔细调整多个统计模型。在这里，我们介绍TweetyNet：一个单一的神经网络模型，它学习如何将鸟鸣的语谱图分割成带注释的音节。我们展示了TweetyNet缓解了依赖分段音频的方法的限制。我们还表明，TweetyNet在来自两种鸣鸟-孟加拉雀和金丝雀-的多个个体上表现良好。最后，我们证明了使用TweetyNet我们可以准确地标注包含多天歌曲的非常大的数据集，并且这些预测的标注复制了行为研究的关键发现。此外，我们还提供了开源软件来帮助其他研究人员，以及可以作为基准的带注释的金丝雀歌曲的大型数据集。我们的结论是，TweetyNet使解决有关BirdSong的一系列新问题成为可能。
Songbirds provide a powerful model system for studying sensory-motor learning. However, many analyses of birdsong require time-consuming, manual annotation of its elements, called syllables. Automated methods for annotation have been proposed, but these methods assume that audio can be cleanly segmented into syllables, or they require carefully tuning multiple statistical models. Here, we present TweetyNet: a single neural network model that learns how to segment spectrograms of birdsong into annotated syllables. We show that TweetyNet mitigates limitations of methods that rely on segmented audio. We also show that TweetyNet performs well across multiple individuals from two species of songbirds, Bengalese finches and canaries. Lastly, we demonstrate that using TweetyNet we can accurately annotate very large datasets containing multiple days of song, and that these predicted annotations replicate key findings from behavioral studies. In addition, we provide open-source software to assist other researchers, and a large dataset of annotated canary song that can serve as a benchmark. We conclude that TweetyNet makes it possible to address a wide range of new questions about birdsong.