III: Medium: VOCAL: Video Organization and Interactive Compositional AnaLytics

摄像头部署通常用于许多应用，如交通监控、动物行为跟踪、自动驾驶、土木工程等。从这些视频流中提取价值是一个关键的研究和商业挑战；一个能够组织并提供用户方便交互和查询大规模视频的界面的系统将在许多商业和学术领域发生变革。然而，开发现代视频应用所需的视频数据管理系统仍处于起步阶段。现有系统有重要的局限性，限制了它们的实际应用：它们不容易适应新的领域；它们仅限于不支持复杂的查询；大多数系统处理来自多个摄像机的视频流，即使这些摄像机是协调部署的一部分。这个项目通过开发voice解决了这些限制：一个用于视频组织和交互式构图分析的开源系统。VOCAL由一套领域无关工具组成，用于端到端视频分析。它支持用户(1)交互式组织视频数据，(2)表达和执行复杂查询，(3)查询多视图摄像机部署。该项目还为本科生和研究生提供研究经验，并为K-12学生提供视频管理和分析方面的教材。为了实现上述目标，本项目在数据库、计算机视觉和人工智能方面贡献了新的方法。它还汇集了这些学科之间的一些独立成果。特别地，VOCAL强调了使用最近的自我监督计算机视觉方法来构建算法的可能性，这些算法可以使数据探索对大型视频数据集可行，从而允许快速发展特定领域的视频事件识别模型。VOCAL还利用场景图表示，允许用户将复杂的查询表示为简单查询的组合。然后，它为交互式规范和有效执行此类查询开发了新的方法。最后，VOCAL提供了无缝查询多视图摄像机部署的新方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Camera deployments are commonly used in many applications such as traffic monitoring, animal behavior tracking, autonomous driving, civil engineering, and more. Extracting value from these video streams is a key research and commercial challenge; a system that can organize and provide an interface for users to easily interact with and query large-scale video is poised to be transformative in many commercial and academic domains. Yet, the video data management systems required to develop modern video applications are still in their infancy. Existing systems have important limitations that restrict their practical use: they do not adapt easily to new domains; they have limited to no support for asking complex queries; and most systems process video streams from multiple cameras independently of one another, even if the cameras are part of a coordinated deployment. This project addresses these limitations by developing VOCAL: an open-source system for Video Organization and Interactive Compositional AnaLytics. VOCAL consists of a suite of domain-agnostic tools for end-to-end video analytics. It supports users with (1) interactively organizing video data, (2) expressing and executing complex queries, and (3) querying multi-view camera deployments. This project also provides research experiences for undergraduate and graduate students and produces materials to teach K-12 students about video management and analytics.To meet the above goals, this project contributes new approaches in databases, computer vision, and AI. It also brings together some of the independent efforts across these disciplines. In particular, VOCAL highlights the possibilities of using recent self-supervised computer vision methods to build algorithms that can make data exploration feasible for large video datasets, and thereby, allowing the rapid development of domain-specific video event recognition models. VOCAL also utilizes scene graph representations to allow users to express complex queries as compositions of simpler ones. It then develops new approaches for the interactive specification and efficient execution of such queries. Finally, VOCAL contributes new approaches to seamlessly querying multi-view camera deployments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.