RI: Small: Collaborative Research: Seeing Surfaces: Actionable Surface Properties from Vision

该项目旨在使计算机和机器人能够从表面的外观（外观）中估计表面的可操作的物理特性（感觉）。其关键思想是利用辐射和物理表面特征之间的深刻交织关系。通过学习模型，使明确的物理表面属性编码在辐射外观属性的全部和部分测量，计算机可以估计关键的物理性质的真实世界的表面从被动观察与新的相机系统。该项目为将这些模型和估计算法集成到场景理解、机器人动作规划和高效视觉传感中铺平了道路。研究结果为计算机视觉提供了一种目前缺失但基本的能力，使计算机视觉，机器人和计算机图形学领域的许多应用受益。该项目为本科生和研究生提供了实践研究的机会，并融入了在德雷克塞尔大学和罗格斯大学教授的PI本科和研究生课程。它们也被用作K-12外展活动的背景，包括高中和中学的导师计划。数据收集活动提供了一个理想的平台，让K-12学生接触物理和计算机science.This研究探讨的方法来推断可操作的表面属性从图像和详细的表面反射率测量。研究活动集中在四个具体目标：1）日常表面的可操作的物理性质和外观测量的受控和非受控的大规模数据收集，2）导出用于从局部表面外观推断物理性质的预测模型，3）包括对象和场景信息的全局语义上下文的集成，以及4）开发有效的外观捕获及其用于新颖的来自外观的物理感测。这些研究成果共同回答了一个基本问题，即计算机视觉如何在不接触表面并不知道它是什么的情况下预测表面的物理特性，为计算视觉行动奠定了基础。

This project is to enable computers and robots the capability of estimating actionable, physical properties of surfaces (the feel) from their appearance (the looks). The key idea is to leverage the deeply interwoven relation between radiometric and physical surface characteristics. By learning models that make explicit the physical surface properties encoded in full and partial measurements of radiometric appearance properties, computers can estimate crucial physical properties of real-world surfaces from passive observations with novel camera systems. This project paves the path for integrating these models and estimation algorithms into scene understanding, robotic action planning, and efficient visual sensing. The research results provide a currently missing but fundamental capability to computer vision that benefits a number of applications in areas of computer vision, robotics, and computer graphics. The project provides hands-on research opportunities for both undergraduate and graduate students and are integrated in the PIs' undergraduate and graduate courses taught at Drexel and Rutgers. They are also used as a backdrop for K-12 outreach activities including high school and middle school mentorship programs. The data collection activities provide an ideal platform to expose K-12 students to physics and computer science.This research investigates the methods to infer actionable surface properties from images and detailed surface reflectance measurements. The research activities are centered on four specific aims: 1) controlled and uncontrolled large-scale data collection of actionable physical properties and appearance measurements of everyday surfaces, 2) derivation of prediction models for deducing physical properties from local surface appearance, 3) integration of global semantic context including object and scene information, and 4) development of efficient appearance capture and its use for novel physics-from-appearance sensing. These research thrusts collectively answer the fundamental question of how computer vision can anticipate the physical properties of a surface without touching it and knowing what it is, laying the foundation for computational vision-for-action.