RI: Small: Collaborative Research: Seeing Surfaces: Actionable Surface Properties from Vision

这个项目是为了使计算机和机器人能够从表面的外观（外观）来估计表面的可操作的物理特性（感觉）。关键思想是利用辐射测量和物理表面特征之间深刻交织的关系。通过学习模型，明确地表物理特性编码在辐射外观特性的全部和部分测量中，计算机可以通过新型相机系统的被动观测来估计现实世界表面的关键物理特性。该项目为将这些模型和估计算法集成到场景理解，机器人行动计划和高效视觉感知中铺平了道路。研究结果为计算机视觉提供了一种目前缺失但基本的能力，有利于计算机视觉、机器人和计算机图形学领域的许多应用。该项目为本科生和研究生提供了动手研究的机会，并被整合到德雷塞尔大学和罗格斯大学的pi本科和研究生课程中。它们也被用作K-12扩展活动的背景，包括高中和初中的指导计划。数据收集活动提供了一个理想的平台，让K-12学生接触物理和计算机科学。本研究探讨了从图像和详细的表面反射率测量中推断可操作的表面特性的方法。研究活动集中在四个具体目标上：1)日常表面可操作的物理特性和外观测量的受控和非受控大规模数据收集；2)推导从局部表面外观推断物理特性的预测模型；3)整合包括对象和场景信息在内的全局语义上下文；4)开发高效的外观捕获及其用于新型外观物理传感。这些研究重点共同回答了计算机视觉如何在不接触表面和不知道它是什么的情况下预测表面的物理特性的基本问题，为计算视觉行动奠定了基础。

This project is to enable computers and robots the capability of estimating actionable, physical properties of surfaces (the feel) from their appearance (the looks). The key idea is to leverage the deeply interwoven relation between radiometric and physical surface characteristics. By learning models that make explicit the physical surface properties encoded in full and partial measurements of radiometric appearance properties, computers can estimate crucial physical properties of real-world surfaces from passive observations with novel camera systems. This project paves the path for integrating these models and estimation algorithms into scene understanding, robotic action planning, and efficient visual sensing. The research results provide a currently missing but fundamental capability to computer vision that benefits a number of applications in areas of computer vision, robotics, and computer graphics. The project provides hands-on research opportunities for both undergraduate and graduate students and are integrated in the PIs' undergraduate and graduate courses taught at Drexel and Rutgers. They are also used as a backdrop for K-12 outreach activities including high school and middle school mentorship programs. The data collection activities provide an ideal platform to expose K-12 students to physics and computer science.This research investigates the methods to infer actionable surface properties from images and detailed surface reflectance measurements. The research activities are centered on four specific aims: 1) controlled and uncontrolled large-scale data collection of actionable physical properties and appearance measurements of everyday surfaces, 2) derivation of prediction models for deducing physical properties from local surface appearance, 3) integration of global semantic context including object and scene information, and 4) development of efficient appearance capture and its use for novel physics-from-appearance sensing. These research thrusts collectively answer the fundamental question of how computer vision can anticipate the physical properties of a surface without touching it and knowing what it is, laying the foundation for computational vision-for-action.