BRC-BIO: Optimizing Snake Identification by Understanding the Interplay of Computer Vision, Crowdsourcing, and Expert Verification

BRC-BIO：通过了解计算机视觉、众包和专家验证的相互作用来优化蛇识别

• 批准号: 2313356
• 负责人: Andrew Durso
• 金额: $ 45.42万
• 依托单位: Florida Gulf Coast University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313356&HistoricalAwards=false
• 关键词: BRC; BIO; Optimizing; Snake; Identification

Species identification is fundamental to biology, and in the case of snakes not only helps understanding their diversity but could also be critical in the case of snake bites. Recently, computer systems that can help people with classification tasks have become a lot more common. Computer vision algorithms are computer programs that have been trained to perform specific classification tasks, such as Shazam, Google Lens, iNaturalist’s Seek, and eBird’s Merlin app. This research aims to understand the influence that conflicting information can have on the accuracy of classification tasks. This is important because it could lead us to an optimal pipeline that allows humans and computers to work together to perform these tasks while making the minimal number of mistakes. This project involves the identification of snake species from photos, a task that is difficult for both humans and computers because there are over 4,000 different snake species and some of them look extremely similar. In the project, people will be shown images of snakes and asked to identify them to species. Sometimes, they will also receive information about what other people or computer vision algorithms called these snakes. The researchers will measure any changes in identification accuracy that are caused by the additional information. The researchers will also gather images of rare species of snakes from preserved specimens in natural history museums to determine whether these types of images can help train computer vision algorithms to better identify these snakes in the wild. This project will provide training opportunities for undergraduate and graduate students, as well as invite participation from the public in identification and data collection. This project will explore the intersection of Convolutional Neural Network (CNN) and human species identification. It will also study effects of accuracy of CNNs from different training sets, using field and collection data. Using a gamified platform, researchers will pre-test the snake ID skills of participants, then randomly assign participants snake ID tasks with or without prior labels. Some prior labels will be correct and others will be incorrect. Some prior labels will appear to come from other participants and others from computer vision algorithms. Researchers will measure how frequently players give an incorrect ID that matches competing or confirming prior information. Whether and how susceptible players are to join the “bandwagon” of incorrect IDs from algorithms or other humans will help determine how and when to add a “human-in-the-loop” while maintaining or increasing identification accuracy. In the second project, the performance of computer vision algorithms trained on images of museum specimens, wild snakes, or both will be compared in order to better understand the potential of using photos of museum specimens to nourish training datasets for computer vision algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

物种鉴定是生物学的基础，对于蛇来说，不仅有助于了解它们的多样性，而且在蛇咬伤的情况下也可能是至关重要的。最近，可以帮助人们完成分类任务的计算机系统变得更加常见。计算机视觉算法是经过训练以执行特定分类任务的计算机程序，如Shazam、Google Lens、iNaturist的Seek和eBird的Merlin应用程序。本研究旨在了解冲突信息对分类任务准确性的影响。这一点很重要，因为它可以将我们引向一条最优的管道，允许人类和计算机一起工作来执行这些任务，同时犯下最少的错误。这个项目涉及从照片中识别蛇的种类，这是一项对人类和计算机都很困难的任务，因为有4000多种不同的蛇，其中一些看起来非常相似。在该项目中，将向人们展示蛇的图像，并要求人们识别它们的物种。有时，他们还会收到其他人或计算机视觉算法所称的这些蛇的信息。研究人员将测量额外信息导致的识别准确性的任何变化。研究人员还将从自然历史博物馆保存下来的标本中收集稀有物种蛇的图像，以确定这些类型的图像是否可以帮助训练计算机视觉算法，以更好地识别野生蛇。该项目将为本科生和研究生提供培训机会，并邀请公众参与身份识别和数据收集。该项目将探索卷积神经网络(CNN)和人类物种识别的交叉。它还将使用现场和收集的数据来研究来自不同训练集的CNN精度的影响。使用游戏化的平台，研究人员将预先测试参与者的蛇ID技能，然后随机分配给参与者有或没有事先标签的蛇ID任务。以前的一些标签将是正确的，而其他标签将是错误的。之前的一些标签似乎来自其他参与者，另一些则来自计算机视觉算法。研究人员将测量玩家给出与竞争或确认先前信息相匹配的错误ID的频率。玩家是否以及在多大程度上会加入来自算法或其他人的错误ID的“大潮”，这将有助于确定如何以及何时添加“人在循环中”，同时保持或提高识别准确性。在第二个项目中，将比较在博物馆标本、野生蛇或两者的图像上训练的计算机视觉算法的性能，以便更好地了解使用博物馆标本的照片为计算机视觉算法提供训练数据集的潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。