RI: Small: A Compositional Approach to Video Segmentation

这个项目正在寻求一种新的视频分割策略，该策略基于将视频分解成多个重叠的像素片段，并随后将这些片段组合成关于视频中是否存在对象的假设。在给定输入视频的情况下，该方法产生一组时空像素区域作为其输出，其中输出区域集与视频中存在的对象具有高度重叠。该项目进一步开发了语义分割、遮挡分析和活动识别的方法，可以利用基于片段的视频表示。该方法的基础是一个称为复合似然的统计框架，它通过变量重叠子集上的低维统计分布隐含地模拟随机向量的联合分布。该统计模型非常适合将视频对象描述为多个重叠片段的集合。使用该框架，正在开发跟踪视频内的重叠片段并生成对象假设的方法。更多的努力旨在提高该方法的计算效率，以解决在线视频分析中的应用。由此产生的算法提高了视频对象分割和跟踪的性能，并为非结构化视频集合(如YouTube上的视频集合)提供了基于内容的视频分类和检索的新方法。该项目正在制作一种新的公开可用的数据集，其中包含细粒度的地面真实视频对象分割，以促进视频分析中的研究活动。该项目与教育相结合，并鼓励高中生在STEM进行研究。

This project is pursuing a novel strategy for video segmentation based on the decomposition of a video into multiple overlapping segments of pixels, and the subsequent composition of these segments into hypotheses about the existence of objects within the video. Given an input video, this approach produces a set of spatio-temporal pixel regions as its output, where the set of output regions has a high degree of overlap with the objects that are present in the video. The project further develops methods for semantic segmentation, occlusion analysis, and activity recognition which can exploit a segment-based video representation. The basis for the approach is a statistical framework known as composite likelihood, which implicitly models the joint distribution of a random vector through distributions of low-dimensional statistics on overlapping subsets of variables. This statistical model is ideally-suited to describing video objects as a collection of multiple overlapping segments. Using this framework, methods are being developed to track overlapping segments within a video and generate object hypotheses. Additional efforts are aimed at improving the computational efficiency of the approach in order to address applications in on-line video analysis.The resulting algorithms yield improved performance in video object segmentation and tracking, and provide new approaches to content-based video categorization and retrieval, for unstructured video collections such as those found on YouTube. The project is producing a novel publicly-available dataset containing fine-grained ground truth video object segmentations, in order to facilitate research activities in video analysis. The project is integrated with education and outreaches high school students to research in STEM.