From Human-Powered to Automated Video Description for Blind and Low Vision Users

针对盲人和低视力用户的从人力到自动视频描述

基本信息

  • 批准号:
    10568469
  • 负责人:
  • 金额:
    $ 64.99万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
  • 财政年份:
    2023
  • 资助国家:
    美国
  • 起止时间:
    2023-07-01 至 2028-06-30
  • 项目状态:
    未结题

项目摘要

Project Summary Approximately 12 million people in the United States have been diagnosed with a visual impairment. These individuals face unique challenges in our modern environment, where much critical information related to education, employment, entertainment, and community is presented in the form of digital videos. Inaccessible information can result in social exclusion or become life threatening if individuals require access to it in order to make decisions related to their health and safety. For example, in a personal or global health crisis, individuals may need to access the mass amounts of information conveyed via videos or dynamic infographics in order to make informed decisions. To address this need, the online platform YouDescribe allows blind and low vision (BLV) users to request amateur volunteers to create video descriptions, also referred to as audio descriptions (AD), of YouTube videos. However, the platform has been unable to keep up with the overwhelming demand, and 92.5% of videos on the YouDescribe user wish list remain undescribed. The overall objective of this proposal is to build an AI-driven system, suitable for use on a wide-scale, to automatically generate descriptions of online videos, as well as answer questions asked by BLV users about the content of videos. The rationale for this project is that AI-based tools are necessary to facilitate timely access to the deluge of new videos appearing on the Internet every day. The proposed work encompasses three specific aims: 1) develop an AI-based tool in collaboration with sighted describers that more efficiently produces video descriptions and increases the availability of accessible videos. The goal is to create an AI-driven NarrationBot that will decrease the time required for novice volunteers to produce video descriptions by 80%; 2) develop an AI-based tool in collaboration with BLV individuals that offers user-driven access to visual information in online videos. The goal is to develop an AI-driven QABot that allows users to pause a video, ask questions about content, and receive immediate answers (e.g., “What breed is the dog?”, “German shepherd”) that are accurate 80% of the time; and 3) develop and publicly release large-scale datasets to improve machine learning for video accessibility. These novel datasets will be used to increase the quality and accuracy of NarrationBot and QABot until AI-generated descriptions and answers need minimal intervention from human volunteers and can serve BLV users directly. The proposed research is innovative because it focuses on videos, whereas existing AI-driven efforts to address this problem have focused primarily on static photos or images. It is also one of only a few efforts to directly partner with BLV individuals to develop AI-driven systems that produce visual descriptions or answer visual questions. The proposed research is significant because it will result in open-source, AI-driven tools that will give BLV individuals unprecedented control over their ability to independently navigate the information-rich world of online videos, thus improving their health and wellbeing.
项目摘要 在美国,大约有1200万人被诊断出患有视力障碍。这些 在我们的现代环境中,个人面临着独特的挑战,许多关键信息与 教育、就业、娱乐和社区的信息以数字视频的形式呈现。不可访问 如果个人需要获取信息, 做出与他们的健康和安全有关的决定。例如,在个人或全球健康危机中, 可能需要访问通过视频或动态信息图表传达的大量信息, 做出明智的决定。为了满足这一需求,在线平台YouDescribe允许盲人和低视力者 (BLV)用户可以请求业余志愿者创建视频描述,也称为音频描述 (AD)YouTube视频。然而,该平台已经无法跟上压倒性的需求, YouDescribe用户愿望清单上92.5%的视频仍然没有描述。本提案的总体目标是 是建立一个人工智能驱动的系统,适用于大规模使用,自动生成在线描述, 视频,以及回答BLV用户关于视频内容的问题。这样做的理由 该项目的一个关键问题是,基于人工智能的工具对于及时访问出现在 每天上网。拟议的工作包括三个具体目标:1)开发一个基于AI的工具, 与有视力的描述者合作,更有效地生成视频描述并增加 提供无障碍视频。我们的目标是创建一个AI驱动的NarrationBot, 要求新手志愿者制作80%的视频描述; 2)合作开发基于AI的工具 与BLV个人,提供用户驱动的访问在线视频中的视觉信息。我们的目标是开发 一个AI驱动的QABot,允许用户暂停视频,询问有关内容的问题, 答案(例如,“狗是什么品种的?”“德国牧羊犬”),准确的80%的时间;和3)发展 并公开发布大规模数据集,以改进机器学习,提高视频可访问性。这些新颖 数据集将用于提高NarrationBot和QABot的质量和准确性,直到AI生成 描述和回答需要人类志愿者的最小干预,并且可以直接为BLV用户服务。 拟议的研究是创新的,因为它专注于视频,而现有的人工智能驱动的努力,以解决 这个问题主要集中在静态照片或图像上。这也是为数不多的直接 与BLV个人合作开发人工智能驱动的系统,以产生视觉描述或回答视觉 问题.拟议的研究意义重大,因为它将产生开源的人工智能驱动的工具, BLV个人对他们独立导航信息丰富的世界的能力的前所未有的控制。 在线视频,从而改善他们的健康和福祉。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Pooyan Fazli其他文献

Pooyan Fazli的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

相似海外基金

TRUST2 - Improving TRUST in artificial intelligence and machine learning for critical building management
TRUST2 - 提高关键建筑管理的人工智能和机器学习的信任度
  • 批准号:
    10093095
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Collaborative R&D
QUANTUM-TOX - Revolutionizing Computational Toxicology with Electronic Structure Descriptors and Artificial Intelligence
QUANTUM-TOX - 利用电子结构描述符和人工智能彻底改变计算毒理学
  • 批准号:
    10106704
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    EU-Funded
Artificial intelligence in education: Democratising policy
教育中的人工智能:政策民主化
  • 批准号:
    DP240100602
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Discovery Projects
Application of artificial intelligence to predict biologic systemic therapy clinical response, effectiveness and adverse events in psoriasis
应用人工智能预测生物系统治疗银屑病的临床反应、有效性和不良事件
  • 批准号:
    MR/Y009657/1
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Fellowship
REU Site: CyberAI: Cybersecurity Solutions Leveraging Artificial Intelligence for Smart Systems
REU 网站:Cyber​​AI:利用人工智能实现智能系统的网络安全解决方案
  • 批准号:
    2349104
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Standard Grant
EAGER: Artificial Intelligence to Understand Engineering Cultural Norms
EAGER:人工智能理解工程文化规范
  • 批准号:
    2342384
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Standard Grant
Reversible Computing and Reservoir Computing with Magnetic Skyrmions for Energy-Efficient Boolean Logic and Artificial Intelligence Hardware
用于节能布尔逻辑和人工智能硬件的磁斯格明子可逆计算和储层计算
  • 批准号:
    2343607
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Standard Grant
I-Corps: Translation Potential of a Secure Data Platform Empowering Artificial Intelligence Assisted Digital Pathology
I-Corps:安全数据平台的翻译潜力,赋能人工智能辅助数字病理学
  • 批准号:
    2409130
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Standard Grant
Planning: Artificial Intelligence Assisted High-Performance Parallel Computing for Power System Optimization
规划:人工智能辅助高性能并行计算电力系统优化
  • 批准号:
    2414141
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Standard Grant
Reassessing the Appropriateness of currently-available Data-set Protection Levers in the era of Artificial Intelligence
重新评估人工智能时代现有数据集保护手段的适用性
  • 批准号:
    23K22068
  • 财政年份:
    2024
  • 资助金额:
    $ 64.99万
  • 项目类别:
    Grant-in-Aid for Scientific Research (B)
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了