RI: Medium: Collaborative Research: Recognition of Materials

我们生活在一个由各种各样的材料组成的世界里，这些材料在外观上的变化丰富了我们的视觉体验。也正是这种材料的可变性增加了图像理解的复杂性。本研究计划旨在为现实世界材料的自动视觉理解和识别建立理论和计算基础。该计划从三个关键方面解决了这一具有挑战性的问题，即，推导1)基于物理和数据驱动的新型混合材料外观的空间、角度、光谱、时间和尺度变化表示，2)用于估计控制材料外观的基于物理参数值的主动和被动方法，3)单图像材料识别方法，利用基于物理的光学参数作为先验或不变量来指导机器学习技术。这些研究重点导致了一套全面的计算工具来识别现实世界图像中的材料，尽管它们的外观变化很复杂，比如识别生锈的金属，从坚硬的混凝土中识别软布，识别牛奶中不同的脂肪含量，以及用软、硬、粗糙和重等材料特征标记图像区域。该项目所产生的能力对于许多场景都至关重要，例如，使人形机器人能够理解它不应该挤压孩子柔软的手，自动驾驶汽车能够理解在崎岖地形中应该避开哪些区域，对组织进行视觉分析以帮助医疗诊断，以及自动检查系统可靠地发现不合格的食品以预防疾病。pi与这些特定应用领域的研究小组合作，紧密地将该项目的结果集成到他们的工作中。这项研究的结果也通过出版物、网站、数据库、新课程和专题讨论会广泛传播。

We live in a world made of diverse materials whose variations in appearance enrich our visual experience. It is also this variability of materials that adds daunting complexity to image understanding. This research program aims to establish the theoretical and computational foundation for automatic visual understanding and recognition of real-world materials. The program tackles this challenging problem from three key aspects, namely, deriving 1) novel hybrid physically-based and data-driven representations of the spatial, angular, spectral, temporal, and scale variations of material appearance, 2) active and passive methods for estimating the values of physically-based parameters that govern material appearance, and 3) single-image material recognition methods that leverage physically-based optical parameters as priors or invariants to guide machine learning techniques. These research thrusts lead to a comprehensive set of computational tools to recognize materials in real-world images despite their complex appearance variations, such as recognizing rusted metals, discerning soft cloth from hard concrete, identifying different fat content of milks, and labeling image regions with material traits like soft, hard, rough, and heavy.The capabilities resulting from this program are crucial to a broad range of scenarios, for instance, to enable humanoid robots to understand that it should not squeeze the soft hands of a child, autonomous vehicles to understand what regions to avoid in a rugged terrain, visual analyses of tissues to help medical diagnosis, and automated inspection systems to reliably discover sub-standard quality food to prevent ill-health. The PIs work with research groups in these specific application areas to closely integrate the results from this project into their efforts. The results from this research are also broadly disseminated via publications, websites, databases, new courses and symposiums.