NRI: Small: Collaborative Research: Active Sensing for Robotic Cameramen

NRI：小型：协作研究：机器人摄影师的主动传感

• 批准号: 1317947
• 负责人: Kostas Daniilidis
• 金额: $ 43.37万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2015-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1317947&HistoricalAwards=false
• 关键词: NRI; Small; Collaborative; Research; Active

With advances in camera technologies, and as cloud storage, network bandwidth and protocols become available, visual media are becoming ubiquitous. Video recording became de facto universal means of instruction for a wide range of applications such as physical exercise, technology, assembly, or cooking. This project addresses the scientific and technological challenges of video shooting in terms of coverage and optimal views planning while leaving high level aspects including creativity to the video editing and post-production stages. Camera placement and novel view selection challenges are modeled as optimization problems that minimize the uncertainty in the location of actors and objects, maximize coverage and effective appearance resolution, and optimize object detection for the sake of semantic annotation of the scene. New probabilistic models capture long range correlations when the trajectories of actors are only partially observable. Quality of potential novel views is modeled in terms of resolution that is optimized by maximizing the coverage of a 3D orientation histogram while an active view selection process for object detection minimizes a dynamic programming objective function capturing the loss due to classification error as well as the resources spent for each view.The project advances active sensing and perception and provides the technology for further automation on video capturing. Such technology has broader impact on the production of education videos for online courses as well as in telepresence applications. Research results are integrated into robotics and digital media programs addressing K-12 students.

随着相机技术的进步，以及云存储、网络带宽和协议的可用，视觉媒体变得无处不在。录像成为事实上的普遍教学手段，广泛应用于体育锻炼、技术、组装或烹饪等领域。该项目解决了视频拍摄在覆盖范围和最佳视图规划方面的科学和技术挑战，同时将包括创意在内的高水平方面留给视频编辑和后期制作阶段。摄像机的位置和新的视图选择的挑战建模为优化问题，最大限度地减少演员和对象的位置的不确定性，最大限度地提高覆盖率和有效的外观分辨率，并优化对象检测的场景的语义注释。新的概率模型捕捉长期的相关性时，演员的轨迹只有部分可观察。 潜在的新视图的质量是根据分辨率来建模的，该分辨率通过最大化3D方向直方图的覆盖范围来优化，而用于对象检测的主动视图选择过程最小化动态规划目标函数，该目标函数捕获由于分类错误而导致的损失以及每个视图所花费的资源。该项目推进了主动传感和感知，并提供了进一步自动化视频捕获的技术。这种技术对在线课程教育视频的制作以及远程呈现应用产生了更广泛的影响。研究成果被整合到机器人和数字媒体方案解决K-12学生。