Unsupervised Machine Learning for Visual Relation Detection

从视频数据中分析和提取有意义的信息是机器学习的一个重要应用，特别是考虑到全球范围内视频的制作、上传、传输和存储的爆炸式增长。 机器学习和最近的深度学习方法在识别静态图像中的对象方面取得了巨大的成功，无论是面部识别，纹理分类，还是在杂乱环境中识别日常物体。 然而，这些方法通常不能很好地推广到视频，我们提出的研究集中在两个挑战： 1.随着时间的推移，物体的位置、形状和相互作用比单个图像中的复杂得多。我们希望推断哪些对象最重要，以及场景中的多个对象如何随着时间的推移相互作用-例如帽子是在人的头上（相互作用），还是在背景中的架子上（非相互作用）。 2.绝大多数视频数据都没有以任何方式进行注释，因此我们的目标是在给定半监督数据（很少注释）或完全无监督数据（没有注释）的情况下推动机器学习的最新技术。 在这里，我们的目标是基于半监督或无监督的方法来分析视频场景，从视频对象分割开始，然后转向更重要的对象交互问题。拟议的研究项目将导致模型学习的改进策略，以及更复杂的视频分析方法，特别是用于理解空间和时间对象关系的高级模型。

The analysis and extraction of meaningful information from video data is a vital application of machine learning, particularly given the explosion of video being produced, uploaded, transmitted, and stored worldwide. Machine learning and, more recently, deep learning methods have shown outstanding success in identifying objects in still images, whether as face recognition, texture classification, or even the recognition of everyday objects in cluttered environments. However these methods typically do not generalize well to video, where our proposed research focuses on two challenges: 1. Object locations, shape, and interactions are much more complex over time than within a single image. We wish to infer which objects are of greatest significance, and how multiple objects in a scene interact with one another over time - for example whether a hat is on a person's head (interacting), or on a shelf in the background (non-interacting). 2. The overwhelming majority of video data is not annotated in any way, so our goal is to push the state-of-the-art in machine learning given semi-supervised data (few annotations) or fully un-supervised data (no annotations). Here we aim to analyze the video scenes based on a semi-supervised or unsupervised approach, starting with video object segmentation, and moving to the much more significant problem of object interaction. The proposed research project will lead to improved strategies for model learning, and to more sophisticated approaches to video analysis, particularly higher-level models for understanding spatial and temporal object relationships.