RI: Small: A Compositional Approach to Video Segmentation

该项目正在寻求一种新颖的视频分割策略，该策略基于将视频分解为多个重叠的像素片段，然后将这些片段组合成有关视频中对象存在的假设。给定输入视频，该方法产生一组时空像素区域作为其输出，其中该组输出区域与视频中存在的对象具有高度重叠。该项目进一步开发了语义分割，遮挡分析和活动识别的方法，可以利用基于段的视频表示。该方法的基础是一个被称为复合似然的统计框架，它通过重叠变量子集上的低维统计分布隐式地模拟随机向量的联合分布。这种统计模型非常适合于将视频对象描述为多个重叠片段的集合。使用这个框架，正在开发的方法来跟踪视频内的重叠段，并生成对象的假设。另外的努力是为了提高计算效率的方法，以解决在线视频analysis.The由此产生的算法产生的视频对象分割和跟踪的性能提高，并提供新的方法，基于内容的视频分类和检索，非结构化的视频收藏，如那些发现在YouTube上的应用。该项目正在制作一个新的公开数据集，其中包含细粒度的地面实况视频对象分割，以促进视频分析的研究活动。该项目与教育相结合，并向高中生提供STEM研究。

This project is pursuing a novel strategy for video segmentation based on the decomposition of a video into multiple overlapping segments of pixels, and the subsequent composition of these segments into hypotheses about the existence of objects within the video. Given an input video, this approach produces a set of spatio-temporal pixel regions as its output, where the set of output regions has a high degree of overlap with the objects that are present in the video. The project further develops methods for semantic segmentation, occlusion analysis, and activity recognition which can exploit a segment-based video representation. The basis for the approach is a statistical framework known as composite likelihood, which implicitly models the joint distribution of a random vector through distributions of low-dimensional statistics on overlapping subsets of variables. This statistical model is ideally-suited to describing video objects as a collection of multiple overlapping segments. Using this framework, methods are being developed to track overlapping segments within a video and generate object hypotheses. Additional efforts are aimed at improving the computational efficiency of the approach in order to address applications in on-line video analysis.The resulting algorithms yield improved performance in video object segmentation and tracking, and provide new approaches to content-based video categorization and retrieval, for unstructured video collections such as those found on YouTube. The project is producing a novel publicly-available dataset containing fine-grained ground truth video object segmentations, in order to facilitate research activities in video analysis. The project is integrated with education and outreaches high school students to research in STEM.