EAPSI: Reconstructing 3D models from 2D images

该奖项支持旨在通过增强的软件和算法开发来简化获得三维模型的过程的研究，从而实现在基本的照相手机或数码相机等日常硬件上创建3D模型。这将允许小型项目执行3D定量研究，而无需定位或投资3D扫描仪。该项目将与台湾国立清华大学赖尚宏博士合作进行。赖博士是计算机视觉领域的专家，他从一张人脸图像中创建了3D人脸模型。类似的原理可以应用于重建3D躯干模型，以加强乳腺癌研究。对3D模型进行表面分析将有助于对乳房形态进行定量分析，有助于改善整形外科技术以调整形状和体积，并在术前咨询期间为患者提供可能的手术结果的3D可视化。在更广泛的计划中，手机应用程序可能会从现场捕获的照片中创建3D模型，这将导致对3D数据的更大访问。将开发一种从多视图图像重建3D模型的算法，用于绘制不同的对象，这将允许3D形状分析。目标是校准图像以确定相机位置，构建目标对象的粗略深度图，并修改建模算法的参数以实现照片一致性。从不同的位置拍摄一个小玩具物体的多张照片，并记录这些位置以供参考。一些挑战是区分阴影边界和结构边缘，并确定光源的方向。图形切割方法将根据图像的轮廓给出3D模型的大致轮廓。利用照明和阴影的其他方法将用于进一步细化3D模型。这个NSF EAPSI奖支持美国研究生的研究，并与台湾科技部合作资助。

This award supports research aimed at simplifying the process of obtaining 3-dimensional models through enhanced software and algorithm development, enabling 3D model creation on everyday hardware such as a basic camera phone or digital camera. This will allow small projects to perform 3D quantitative studies without having to locate or invest in a 3D scanner. This project will be conducted in collaboration with Dr. Shang-Hong Lai of National Tsing-Hua University in Taiwan. Dr. Lai, an expert in the field of computer vision, has created 3D face models from a single face image. Similar principles can be applied to reconstruct 3D torso models to enhance breast cancer research. Performing surface analysis on the 3D models will facilitate the quantitative analysis of the breast morphology, help improve plastic surgery techniques to adjust shape and volume, and provide 3D visualization of possible surgical outcomes to patients during their pre-surgical consultation. In the broader scheme, a cell phone app could potentially create a 3D model from captured photos on the scene, which will lead to a greater accessibility to 3D data. An algorithm on reconstructing 3D models from multi-view images will be developed for rendering different objects, which will allow for 3D shape analysis. The objectives are to calibrate the images to determine camera positions, construct a coarse depth map of the target object, and modify the parameters of the modeling algorithm to achieve photo-consistency. Multiple photographs of a small toy object will be taken from various positions and these positions will be recorded for reference. Some challenges are differentiating shadow boundaries from structural edges and determining the directions of the light sources. The graph cut method will give the general outline of the 3D model based on the silhouettes from the images. Other methods utilizing illumination and shading will be used to further refine the 3D model. This NSF EAPSI award supports the research of a U.S. graduate student and is funded in collaboration with the Ministry of Science and Technology of Taiwan.