RI: Small: Recovering Object 3D Shape and Material from Isolated Images

该项目提高了计算机从视觉传感器解释物体形状和材料的能力。 该研究假设，完整的3D物体形状可以通过将观察到的物体的视觉特征与来自数据集的已知形状的物体相匹配，转移已知形状，并使3D形状变形以更好地考虑匹配特征的空间对应性来估计。 该研究在多个尺度上代表材料，分别编码材料类别模式中的小凸起和凹槽。 由于图像属性来自形状，材料和照明的组合，因此研究还涉及开发联合估计的算法。所开发的技术可应用于自动化系统、个人和工业机器人、监控和安全、运输、图像检索、图像编辑和操作以及内容创建。该项目通过学生项目、课程开发以及涉及更广泛受众的讲习班和辅导，为教育做出贡献。该研究探讨了改进的三维形状和材料的表示和方法，以恢复他们从一个图像。研究团队并不以精确的表面法线或BRDF参数等真实模型为目标，而是恢复对对象识别、内容创建和其他任务有用的近似模型。 3D物体形状的工作重点是将物体边界标记为遮挡、折叠或纹理/纹理，并将这些边界作为数据驱动方法的一部分来恢复物体的完整3D模型。 该研究涉及研究恢复组成物体的材料的丰富，多尺度表示的方法。 这些方法利用近似形状表示和近似表示的照明恢复估计的辐射特性的对象在一个点。 这些算法构建了这些材料属性的地图，以模拟大理石中的空间变化和复杂现象，如大理石中的纹理。 该研究还涉及扩展这些方法来报告空间变化的法线贴图，这些法线贴图捕获了树皮等形状纹理。 最后，研究如何将图像为中心的地图，以捕捉更多的随机，空间局部化的现象，如橙子皮的坑。

This project improves a computer's ability to interpret the shape and material of objects from visual sensors. The research hypothesizes that full 3D object shape can be estimated by matching visual features from an observed object to an object of known shape from a dataset, transferring the known shape, and deforming the 3D shape to better account for spatial correspondences of matched features. The research represents materials at multiple scales, separately encoding little bumps and grooves from the patterns of material categories. Because image properties arise from the combination of shape, material, and illumination, the research also involves developing algorithms to jointly estimate. The developed technologies can be applied to automated systems, personal and industrial robotics, surveillance and security, transportation, image retrieval, image editing and manipulation, and content creation. The project contributes to education through student projects, course development, and workshops and tutorials involving a broader audience. The research investigates improved representations of 3D shape and material and methods to recover them from one image. Rather than aiming for veridical models, such as precise surface normals or BRDF parameters, the research team recovers approximate models that are useful for object recognition, content creation, and other tasks. The work on 3D object shape focuses on labeling object boundaries as occlusions, folds, or texture/albedo and using these boundaries as part of a data-driven approach to recover full 3D models of the objects. The research involves studying methods to recover rich, multiscale representations of the materials that compose objects. These methods exploit approximate shape representations and approximate representations of the illumination to recover estimates of radiometric properties of the object at a point. The algorithms build maps of these material properties to model spatial variation in albedo and complex phenomena like veins in marble. The research also involves extending these methods to report spatially varying normal maps that capture shape textures like the bark of trees. Finally, the research investigates how to incorporate image-centered maps to capture more random, spatially localized phenomena like the pits in orange peel.