CAREER: Data Management for Exploratory Video Analytics

在过去的十年里，计算机视觉经历了翻天覆地的变化，出现了能够处理各种视觉任务的机器学习模型，比如物体检测和动作定位。然而，这些模型需要花费大量时间来处理视频，并且利用它们来分析视频涉及到大量的编程工作。数据库界对设计视频数据库管理系统以解决这些效率和可用性挑战越来越感兴趣。该项目寻求开发新的技术来加快对大型视频数据集的查询，例如检索导致足球比赛触地得分的游戏。它将减少在从城市规划到天体物理学等广泛应用中分析视频的人力成本。将寻求与领域科学家和行业从业者的密切合作，以帮助将这些想法转化为科学和企业应用。拟议的工作包括在佐治亚理工学院提供的数据库课程中与视频数据库系统有关的综合教育计划。该项目从新的角度解决加速视频分析的突出问题。最近提出的视频数据库系统有两个关键限制。首先，它们的优化器不是为离线探索性视频分析量身定做的，这会大规模浪费系统资源，并增加查询处理的成本。其次，它们支持与检测对象相关的有限范围的查询，并且无法支持更丰富的查询，如本地化操作或重新标识对象。这项提议的目标是解决最先进的视频数据库系统的这些局限性。这项工作将产生关于如何为复杂的视觉管道构建精度驱动的优化的知识。它将通过开发生成合成数据集的技术，解决由于隐私问题而导致的大型数据集不足的问题。创新不在于创建新的VISION模型，而在于新的数据库风格的技术，以更好地优化探索性查询工作负载，并扩大视频数据库系统支持的查询范围。总体而言，这项工作在数据库系统领域进行了创新，开启了优化探索性视频分析的新研究路线。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer vision has gone through a seismic shift over the last decade, with the availability of machine learning models capable of tackling diverse vision tasks like object detection and action localization. However, these models take a lot of time to process a video, and leveraging them for analyzing videos involves a non-trivial programming effort. There is a growing interest in the database community in designing video database management systems to tackle these efficiency and usability challenges. This project seeks to develop novel techniques for speeding up queries over large video datasets like retrieving the gameplays that lead to a touchdown in a football game. It will reduce the human labor cost of analyzing videos in a wide range of applications, ranging from urban planning to astrophysics. A close collaboration with domain scientists and industry practitioners will be pursued to help transfer the ideas to scientific and enterprise applications. The proposed work includes an integrated education plan related to video database systems in the database courses offered at Georgia Tech.This project tackles the outstanding problem of accelerating video analytics from a new standpoint. Recently proposed video database systems have two key limitations. First, their optimizers are not tailored for offline, exploratory video analytics, wasting system resources at scale and raising the cost of query processing. Second, they support a limited range of queries related to detecting objects and are unable to support richer queries like localizing actions or re-identifying objects. The goal of this proposal is to address these limitations of state-of-the-art video database systems. This work will generate knowledge of how to construct accuracy-driven optimization for complex vision pipelines. It will address the paucity of large datasets due to privacy concerns, by developing techniques for generating synthetic datasets. The innovation is not in creating new vision models, but in new database-style techniques to better optimize exploratory query workloads and to broaden the range of queries supported by video database systems. Overall, this work innovates upon the field of database systems to start a new line of research on optimizing exploratory video analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.