NRI: Small: Collaborative Research: Active Sensing for Robotic Cameramen

随着摄像头技术的进步，以及云存储、网络带宽和协议的可用，视觉媒体正变得无处不在。录像实际上成为了广泛应用的通用教学手段，如体育锻炼、技术、组装或烹饪。该项目解决了视频拍摄在复盖面和最佳视角规划方面的科学和技术挑战，同时将包括创造性在内的高层次方面留给了视频编辑和后期制作阶段。将摄像机放置和新颖的视角选择问题建模为优化问题，最小化演员和对象位置的不确定性，最大化覆盖范围和有效的外观分辨率，并优化对象检测以实现场景的语义标注。当参与者的轨迹只有部分可见时，新的概率模型捕捉到了长范围的关联。潜在新视图的质量根据分辨率来建模，该分辨率通过最大化3D方向直方图的覆盖率来优化，而用于目标检测的主动视图选择过程最小化了动态规划目标函数，该目标函数捕捉由于分类错误而造成的损失以及每个视图所花费的资源。该项目促进了主动感知和感知，并为视频捕获的进一步自动化提供了技术。这种技术对在线课程和网真应用程序的教育视频制作产生了更广泛的影响。研究成果被整合到面向K-12学生的机器人和数字媒体项目中。

With advances in camera technologies, and as cloud storage, network bandwidth and protocols become available, visual media are becoming ubiquitous. Video recording became de facto universal means of instruction for a wide range of applications such as physical exercise, technology, assembly, or cooking. This project addresses the scientific and technological challenges of video shooting in terms of coverage and optimal views planning while leaving high level aspects including creativity to the video editing and post-production stages. Camera placement and novel view selection challenges are modeled as optimization problems that minimize the uncertainty in the location of actors and objects, maximize coverage and effective appearance resolution, and optimize object detection for the sake of semantic annotation of the scene. New probabilistic models capture long range correlations when the trajectories of actors are only partially observable. Quality of potential novel views is modeled in terms of resolution that is optimized by maximizing the coverage of a 3D orientation histogram while an active view selection process for object detection minimizes a dynamic programming objective function capturing the loss due to classification error as well as the resources spent for each view.The project advances active sensing and perception and provides the technology for further automation on video capturing. Such technology has broader impact on the production of education videos for online courses as well as in telepresence applications. Research results are integrated into robotics and digital media programs addressing K-12 students.