BIGDATA: Collaborative Research: IA: BirdVox: Automating Acoustic Monitoring of Migrating Bird Species

BIGDATA：协作研究：IA：BirdVox：迁徙鸟类的自动声学监测

• 批准号: 1633259
• 负责人: Juan Bello
• 金额: $ 61.24万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1633259&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; IA; BirdVox

Current bioacoustic monitoring of natural environments requires processing by humans to extract information content from recordings. Thus human processing creates a fundamental bottleneck in which data collection far outpaces capabilities to extract relevant and desired information. Bioacoustic research on automatic species classification in natural environments can be broadly divided into two groups: distinguishing a predefined set of known species from audio clips and extracting species as events that occur in a continuous audio stream. Both classification techniques have their specific problems--many of the data used distinguishing predefined species are recorded under "studio" conditions and not extensible to natural conditions, while processing of continuous audio streams generate many false positives. To overcome these challenges we will take a multi-tiered approach: Analyzing a data set consisting of full-night recordings from 10 recording units over 100 nights. Building a web-enabled software to engage citizen scientists to identify the flight calls, providing us with a large and extensive model training dataset. Developing novel convolutional deep-learning networks, which are well suited for analysis of complex auditory scenes. Visualizing patterns detected and classified flight calls in space and time to produce novel information about the bird migration. Comparing model-generated acoustic data with radar, video, and direct visual citizen science datasets to produce the most comprehensive accounts of nocturnal bird migration possible. The combination of domain knowledge in bird vocalizations, engaging citizen scientists to allow development of large well annotated training datasets, and taking a novel deep-learning approach, will finally resolve the machine classification of acoustic signals in natural environments.

当前对自然环境的生物声学监测需要人类处理以从记录中提取信息内容。因此，人类的处理创造了一个根本的瓶颈，其中数据收集远远超过了提取相关和所需信息的能力。在自然环境中自动物种分类的生物声学研究可以大致分为两组：从音频剪辑中区分一组预定义的已知物种和提取物种作为发生在连续音频流中的事件。这两种分类技术都有其特定的问题--许多用于区分预定义物种的数据是在“工作室”条件下记录的，不能扩展到自然条件，而连续音频流的处理会产生许多误报。为了克服这些挑战，我们将采取多层次的方法：分析由10个记录单元在100个夜晚的整晚记录组成的数据集。构建一个支持网络的软件，让公民科学家来识别航班呼叫，为我们提供一个庞大而广泛的模型训练数据集。 开发新型卷积深度学习网络，非常适合分析复杂的听觉场景。 可视化模式检测和分类飞行呼叫的空间和时间，以产生新的信息有关的鸟类迁徙。 将模型生成的声学数据与雷达，视频和直接视觉公民科学数据集进行比较，以产生最全面的夜间鸟类迁徙数据。 结合鸟类发声的领域知识，让公民科学家参与开发大型注释良好的训练数据集，并采取新的深度学习方法，最终将解决自然环境中声学信号的机器分类问题。

项目成果

Adaptive Pooling Operators for Weakly Labeled Sound Event Detection

• DOI: 10.1109/taslp.2018.2858559
• 发表时间: 2018-04
• 期刊: IEEE/ACM Transactions on Audio, Speech, and Language Processing
• 作者: Brian McFee;J. Salamon;J. Bello

Scaper: A library for soundscape synthesis and augmentation

• DOI: 10.1109/waspaa.2017.8170052
• 发表时间: 2017-10
• 期刊: 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)
• 作者: J. Salamon;D. MacConnell;M. Cartwright;P. Li;J. Bello

Hybrid scattering-LSTM networks for automated detection of sleep arousals

• DOI: 10.1088/1361-6579/ab2664
• 发表时间: 2019-07-01
• 期刊: PHYSIOLOGICAL MEASUREMENT
• 影响因子: 3.2
• 作者: Warrick, Philip A.;Lostanlen, Vincent;Homsi, Masun Nabhan
• 通讯作者: Homsi, Masun Nabhan

Matching human vocal imitations to birdsong: An exploratory analysis

将人类声音模仿与鸟鸣相匹配：探索性分析

• 发表时间: 2019
• 期刊: Animals and Robots
• 作者: Oudyk, K;Lostanlen, V;Salamon, J;Farnsworth, A;Bello, JP
• 通讯作者: Bello, JP

Kymatio: Scattering transforms in Python

Kymatio：Python 中的散射变换

• 发表时间: 2020
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Andreux, Mathieu;Angles, Tomás;Exarchakis, Georgios;Leonarduzzi, Roberto;Rochette, Gaspar;Thiry, Louis;Zarka, John;Mallat, Stéphane;Andén, Joakim;Belilovsky, Eugene
• 通讯作者: Belilovsky, Eugene