NRI: Small: Collaborative Research: Active Sensing for Robotic Cameramen

随着摄像技术的进步，以及云存储、网络带宽和协议的可用性，视觉媒体变得无处不在。视频录制实际上成为了广泛应用的通用教学手段，如体育锻炼、技术、装配或烹饪。该项目解决了视频拍摄在覆盖范围和最佳视角规划方面的科技挑战，而将创意等高层次方面留给视频编辑和后期制作阶段。摄像机放置和新视角选择挑战被建模为优化问题，最小化演员和物体位置的不确定性，最大化覆盖范围和有效的外观分辨率，并优化物体检测以实现场景的语义注释。当参与者的轨迹只能部分观察到时，新的概率模型捕捉到长期的相关性。潜在新视图的质量根据分辨率建模，该分辨率通过最大化3D方向直方图的覆盖范围来优化，而用于对象检测的主动视图选择过程最大限度地减少了动态规划目标函数，该函数捕获了由于分类错误造成的损失以及每个视图所花费的资源。该项目推进了主动传感和感知，并为视频捕获的进一步自动化提供了技术。这种技术对在线课程和远程呈现应用的教育视频制作产生了更广泛的影响。研究成果被整合到面向K-12学生的机器人和数字媒体项目中。

With advances in camera technologies, and as cloud storage, network bandwidth and protocols become available, visual media are becoming ubiquitous. Video recording became de facto universal means of instruction for a wide range of applications such as physical exercise, technology, assembly, or cooking. This project addresses the scientific and technological challenges of video shooting in terms of coverage and optimal views planning while leaving high level aspects including creativity to the video editing and post-production stages. Camera placement and novel view selection challenges are modeled as optimization problems that minimize the uncertainty in the location of actors and objects, maximize coverage and effective appearance resolution, and optimize object detection for the sake of semantic annotation of the scene. New probabilistic models capture long range correlations when the trajectories of actors are only partially observable. Quality of potential novel views is modeled in terms of resolution that is optimized by maximizing the coverage of a 3D orientation histogram while an active view selection process for object detection minimizes a dynamic programming objective function capturing the loss due to classification error as well as the resources spent for each view.The project advances active sensing and perception and provides the technology for further automation on video capturing. Such technology has broader impact on the production of education videos for online courses as well as in telepresence applications. Research results are integrated into robotics and digital media programs addressing K-12 students.