RI: Small: Engineering and Learning Visual Representations

视觉数据，包括视频图像，传达了关于场景中感兴趣的物体的“信息”：形状、材料、身份、关系等。然而，它也是高度冗余的，并且受到变化的影响，这与感兴趣的场景的属性无关，而是取决于传感器，有利位置和光源的性质等。该项目解决了确定应该推断和存储成像数据的什么功能的问题，即对于诸如物体或场景检测、定位、识别和分类等一类任务，尽可能具有“信息性”，同时尽可能“压缩”，并且对讨厌的变异性不敏感。这样的函数称为表示。本研究具有教学价值，通过将看似不相关的方法构建为理想表示的不同近似，从而促进了计算机视觉的教育过程。这进一步有望促进设计更好的表示，从而改进视觉识别（检测、定位、识别和分类）系统的算法，对从自主（例如机器人导航和监视）到交互（例如辅助手术和增强现实）的一系列应用产生影响。该项目根据信息瓶颈原则构建了推断最佳任务特定表示的问题，并在此框架内解决了可计算性、近似值和降维问题。它还解决了“可学习性”问题，以确定通用学习体系结构是否可以近似于最佳表示。信息瓶颈是最小充分统计量概念的泛化和放松，其中明确考虑了复杂性约束和任务相关性。挑战在于，可视化数据的生成过程建模需要复杂的几何（表面形状）、拓扑（遮挡）、光度（材料反射、照明）和动态（运动），其中感兴趣的对象生活在无限维空间中。因此，信息瓶颈甚至很难形式化，更不用说实例化、计算和优化了。该项目的重点是开发易于处理且具有性能保证的信息瓶颈近似值。

Visual data, including video imagery, conveys "information" about objects of interest within the scene: Shape, material, identity, relations, etc. However, it is also highly redundant, and subject to variability that has little to do with the properties of the scene of interest, but instead depend on the sensor, the vantage point, and the nature of the illuminant, etc. This project addresses the question of determining what function of imaging data should be inferred and stored, that is, as "informative" as possible for a class of tasks such as object or scene detection, localization, recognition and categorization, and at the same time as "compressed" as possible, and insensitive to nuisance variability. Such a function is called a Representation. This research has pedagogical value, by framing seemingly unrelated methods as different approximations of an ideal Representation, thus facilitating the educational process in Computer Vision. This is further expected to facilitate the design of better Representations, and therefore improved algorithms for visual recognition (detection, localization, recognition, and categorization) systems, with impact in a range of applications from autonomy (e.g., robotic navigation and surveillance) to interaction (e.g., assisted surgery and augmented reality).The project frames the problem of inferring optimal task-specific Representations in terms of the Information Bottleneck Principle, and addresses issues of computability, approximation, and dimensionality reduction within this framework. It also addresses questions of "learnability," to determine whether a generic learning architecture can approximate an optimal representation. The Information Bottleneck is a generalization and relaxation of the notion of minimal sufficient statistic, where complexity constraints and task relevance are explicitly taken into account. The challenge is that modeling the generative process for visual data entails complex geometry (surface shape), topology (occlusions), photometry (material reflection, illumination), and dynamics (motion) with the object of interest living in infinite-dimensional spaces. Thus, the Information Bottleneck is difficult to even formalize, let alone instantiate, compute, and optimize. The project focuses on developing approximations of the Information Bottleneck that are tractable and yet enjoy performance guarantees.