CCRI: Medium: Developing a Multi-Channel Naturalistic Audio Corpora for the Natural Language Processing Research Community

点击翻译按钮获取中文摘要

This project is focused on developing a massive audio resource of real-world speech communications related to task solving by engineers/scientists working to ensure the success of one of history’s greatest scientific and technical accomplishments - the NASA Apollo missions. Major challenges exist in exploring team based voice communications due to a lack of any comprehensive time-synchronized and transcribed audio resource. The research team will create the framework needed to recover and make available the entire team communications. This includes digitizing the extensive 50-year-old Apollo 30-track analog tape collection, and advance the necessary speech technology tools/resources to automatically generate meta-data that includes when speech occurs, all text transcripts, speaker identity, as well as aspects relating to speaker traits. The resource is expected to encompass 150,000 hours of audio from all Apollo missions. The research team will provide open user access to explore and navigate the community resource using an interactive web platform: Explore Apollo, as well as download audio and corresponding meta-data: Fearless Steps – Explore Apollo. ‘Finding Waldo’: a resource to identify all instances of individual NASA members across Apollo missions and to make this available to surviving personnel and family members as a tribute to the ‘Heroes behind the Heroes of Apollo’. The focused research communities include speech technology, speech and language communications, team psychology in social sciences, education/STEM, historians and preservation archivists. Outreach efforts will be through mini-workshops, tutorials, community challenges, and special sessions across various fields, providing opportunities to distribute and receive user feedback to enhance the resource. This resource will allow engineers, scientists, educators and historians unique data to develop new theories and models for how people work and respond rapidly to challenging problems, as well as promote science and math based (STEM) goals for space, history and team based learning.This project is focused on developing a massive audio resource of real-world team-based speech communications by engineers/scientists working to ensure the success of one of history’s greatest technical achievements, the NASA Apollo missions. There is significant need from the speech technology community for access to natural big-data speech corpora to develop next generation technologies. A critical challenge is the ability to employ audio that is team and task based and not simulated. This project will establish a sustainable multi-speaker task-based corpora generation process based on the recovery of Apollo missions, encompassing up to +150,000 hours of audio. Research activities include (i) establishing the framework needed to digitize the 50-year old Apollo 30-track analog tape collection, (ii) advance speech technology tools/resources to automatically generate meta-data that include speech activity, speech recognition transcript generation, speaker identity, as well as aspects relating to speaker traits. Specific advancements will address acoustic and expanded lexicon/language model requirements to encompass communication traits for NASA engineers. The research team will provide open user access to explore and navigate the community resource using an interactive web platform: Explore Apollo, and download audio and meta-data: Fearless Steps – Explore Apollo. The resource is significantly enhanced by advancing extensive machine learning speech technologies in transcript/meta-data generation for audio speaker diarization – the process of determining “who spoke, what, and when”. The technology offers a unique opportunity to provide portions of history, and tangible pieces of technology for multi-purpose use. The open-access resource provides freely available meta-data to propose and develop algorithms for speech activity detection, keyword spotting, speaker variability, sentiment, accent, language identification, multimodal systems, conversational analysis, speaker turn detection and individual as well as team assessment. The concept of ‘Where’s Waldo’ is used as a metaphor to pay tribute and yield personal recognition to the thousands of notable members across the Apollo missions in addition to using deep learning strategies to develop effective speaker tagging and hot-spot detection systems. This will impact the lives of the Apollo members and their families and provide additional education resources for future generations while also aiding as a historical archive.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.