RI: Medium: Collaborative Research: Recognition of Materials

我们生活在一个由各种材料组成的世界里，这些材料的外观变化丰富了我们的视觉体验。材料的这种可变性也给图像理解增加了令人生畏的复杂性。该研究项目旨在为现实世界材料的自动视觉理解和识别建立理论和计算基础。该计划从三个关键方面解决了这一具有挑战性的问题，即：1）材料外观的空间，角度，光谱，时间和尺度变化的新型混合物理和数据驱动表示，2）用于估计控制材料外观的物理参数值的主动和被动方法，以及3）利用基于物理的光学参数作为先验或不变量来指导机器学习技术的单图像材料识别方法。这些研究推动了一套全面的计算工具来识别真实世界图像中的材料，尽管它们具有复杂的外观变化，例如识别生锈的金属，从坚硬的混凝土中辨别软布，识别牛奶的不同脂肪含量，以及用软，硬，粗糙和沉重等材料特性标记图像区域。例如，使人形机器人能够理解它不应该挤压儿童柔软的手，自动驾驶汽车能够理解在崎岖的地形中应该避开哪些区域，对组织进行视觉分析以帮助医疗诊断，以及自动检测系统能够可靠地发现质量不合格的食品以防止疾病。PI与这些特定应用领域的研究小组合作，将该项目的结果紧密结合到他们的工作中。这项研究的结果还通过出版物、网站、数据库、新课程和研讨会广泛传播。

We live in a world made of diverse materials whose variations in appearance enrich our visual experience. It is also this variability of materials that adds daunting complexity to image understanding. This research program aims to establish the theoretical and computational foundation for automatic visual understanding and recognition of real-world materials. The program tackles this challenging problem from three key aspects, namely, deriving 1) novel hybrid physically-based and data-driven representations of the spatial, angular, spectral, temporal, and scale variations of material appearance, 2) active and passive methods for estimating the values of physically-based parameters that govern material appearance, and 3) single-image material recognition methods that leverage physically-based optical parameters as priors or invariants to guide machine learning techniques. These research thrusts lead to a comprehensive set of computational tools to recognize materials in real-world images despite their complex appearance variations, such as recognizing rusted metals, discerning soft cloth from hard concrete, identifying different fat content of milks, and labeling image regions with material traits like soft, hard, rough, and heavy.The capabilities resulting from this program are crucial to a broad range of scenarios, for instance, to enable humanoid robots to understand that it should not squeeze the soft hands of a child, autonomous vehicles to understand what regions to avoid in a rugged terrain, visual analyses of tissues to help medical diagnosis, and automated inspection systems to reliably discover sub-standard quality food to prevent ill-health. The PIs work with research groups in these specific application areas to closely integrate the results from this project into their efforts. The results from this research are also broadly disseminated via publications, websites, databases, new courses and symposiums.