View Synthesis for Dynamic Scenes, With and Without Reconstruction

给出了一组在不同位置和不同时间拍摄的照片，研究了合成包含多个运动中刚性物体的动态三维场景的新视图的问题。开发了基于图像的内插和外推方法，以便可以合成物理上正确的运动序列，以显示场景对象在缺失的时间间隔内如何移动。将使用与对象可以经历的基本类型的刚性运动(例如，平移运动或绕单个轴旋转)相对应的运动类别。对于每个课程，将设计、实施和测试在不首先恢复场景几何图形的情况下合成物理正确视图的技术。作为基于非重建图像的视图和运动合成的替代方案，还将探索利用对象运动信息来执行相机自校准和3D场景重建的方法。通过恢复每个运动对象的消失点和基本矩阵等属性，可以直接计算摄像机校准信息，而不需要点对应。结果将包括对基于图像的动态场景方法的更好的理论理解，以及用于运动内插和多视点、基于运动的场景重建的新算法。

This research investigates the problem of synthesizing novel views of dynamic, three-dimensional scenes containing multiple rigid objects in motion, given a set of photographs captured at different positions and different times. Image-based interpolation and extrapolation methods are developed so that physically-correct motion sequences can be synthesized showing how the scene objects could have moved during the missing time interval. Motion classes corresponding to basic types of rigid motion an object can undergo (e.g., translational motion or rotation about a single axis) will be used. For each class, techniques for synthesizing physically-correct views without first recovering the scene geometry, will be designed, implemented and tested. As an alternative to non-reconstructive image-based view and motion synthesis, methods that utilize object motion information to perform camera self-calibration and 3D scene reconstruction will also be explored. By recovering properties such as the vanishing point and the fundamental matrix for each moving object, camera calibration information can be directly computed without necessitating point correspondences. Results will include a better theoretical understanding of image-based methods for dynamic scenes, and new algorithms for motion interpolation and multi-view, motion-based scene reconstruction.