I-Corps: Transforming 2D Video into an Interactive 3D Viewing Experience

提出了一种将长视频序列从单摄像机转换为三维全景可视化的新方法。这种方法有望在笔记本电脑上实现近乎实时的工作。3D内容允许用户改变视角，它还支持全景视图。该系统由两个实时软件组成：(1)图像匹配器，从二维图像序列生成多视图全景拼接；以及(2)可使用市售3D显示器向最终用户提供场景的虚拟飞行或演练的3D渲染器。该方法有望实现各种相机类型（高端/低端彩色，红外，甚至伽马射线/r射线成像），相机运动（线性，圆形），运动平台（机载，地面车辆，手持）和场景距离（从几英寸到几千英尺）的实时3D可视化。这项技术已经从环境监测、安全和国防应用的研究中成熟起来，在这些应用中，视频是从飞机或地面车辆上的单个摄像机捕获的。该系统能够生成视频的3D可视化，用于进一步的视频数据分析、场景理解和科学发现。该技术不仅应用于国防和环境监测，还应用于娱乐、医疗成像、检验、房地产和土地调查/开发等行业。从电视到智能手机，消费市场上已经有很多3D设备。但内容仍然缺失。为了解决这一未满足的需求，该团队正在构建一个智能手机应用程序，使用户能够生成3D内容，而不需要3D相机。

A novel method is developed to convert long video sequences from a single camera into 3D panoramic visualizations. The approach holds the promise to work at near real-time on a laptop. The 3D content allows users to change view angles and it also supports panoramic views. The system has two real-time software components: (1) the image matcher that generates multi-view panoramic mosaics from 2D image sequences; and (2) the 3D renderer that can provide an end user a virtual fly-through or walk-through of a scene using commercially available 3D displays. The approach promises real-time 3D visualization for a broad range of camera types (high end/low end color, IR, even gamma-ray/r-ray imaging), camera motions (linear, circular), motion platforms (airborne, ground vehicle, hand-held), and scene distances (from a few inches to thousand feet). This technology has matured from research into environmental monitoring, security and defense applications, where video was captured from a single camera on board an aircraft or ground vehicle. The system is able to produce 3D visualizations of the videos for further video data analysis, scene understanding and scientific discovery. The technology has applications not only in defense and environmental monitoring, but also in entertainment, medical imaging, inspection, real estate and land survey/development industries. There are many 3D ready devices, from TVs, to smartphones in the consumer market already. But the content is still missing. To address this unmet need, the team is building a smartphone application to enable user-generated 3D content, without requiring a 3D camera.