RI: Medium: Collaborative Research: Recognition of Materials

我们生活在一个由不同材料组成的世界，这些材料的外观变化丰富了我们的视觉体验。也正是这种材料的多样性给图像理解增加了令人望而生畏的复杂性。这项研究计划旨在为自动视觉理解和识别真实世界的材料建立理论和计算基础。该程序从三个关键方面解决了这一具有挑战性的问题，即：1)材料外观的空间、角度、光谱、时间和尺度变化的新颖的基于物理和数据驱动的混合表示；2)用于估计控制材料外观的基于物理的参数值的主动和被动方法；以及3)利用基于物理的光学参数作为先验或不变量来指导机器学习技术的单图像材料识别方法。这些研究努力导致了一套全面的计算工具来识别真实世界图像中的材料，尽管它们的外观复杂多变，例如识别生锈的金属，区分坚硬的混凝土中的软布，识别牛奶的不同脂肪含量，以及使用软、硬、粗糙和重等材料特性标记图像区域。该程序产生的能力对于广泛的场景至关重要，例如，使类人机器人了解它不应该挤压孩子的软手，使自动车辆了解在崎岖的地形中应避开哪些区域，对组织进行视觉分析以帮助医疗诊断，以及自动检测系统，以可靠地发现质量不合格的食品，以防止不健康。私人投资机构与这些特定应用领域的研究小组合作，将该项目的成果紧密地整合到他们的工作中。这项研究的结果还通过出版物、网站、数据库、新课程和研讨会广泛传播。

We live in a world made of diverse materials whose variations in appearance enrich our visual experience. It is also this variability of materials that adds daunting complexity to image understanding. This research program aims to establish the theoretical and computational foundation for automatic visual understanding and recognition of real-world materials. The program tackles this challenging problem from three key aspects, namely, deriving 1) novel hybrid physically-based and data-driven representations of the spatial, angular, spectral, temporal, and scale variations of material appearance, 2) active and passive methods for estimating the values of physically-based parameters that govern material appearance, and 3) single-image material recognition methods that leverage physically-based optical parameters as priors or invariants to guide machine learning techniques. These research thrusts lead to a comprehensive set of computational tools to recognize materials in real-world images despite their complex appearance variations, such as recognizing rusted metals, discerning soft cloth from hard concrete, identifying different fat content of milks, and labeling image regions with material traits like soft, hard, rough, and heavy.The capabilities resulting from this program are crucial to a broad range of scenarios, for instance, to enable humanoid robots to understand that it should not squeeze the soft hands of a child, autonomous vehicles to understand what regions to avoid in a rugged terrain, visual analyses of tissues to help medical diagnosis, and automated inspection systems to reliably discover sub-standard quality food to prevent ill-health. The PIs work with research groups in these specific application areas to closely integrate the results from this project into their efforts. The results from this research are also broadly disseminated via publications, websites, databases, new courses and symposiums.