Collaborative Research: CNS Core: Medium: Panoptes: Next Generation Multi-Perspective Video Delivery at Internet Scale

流视频的成功已经引起了对更新形式的多视角视频内容的兴趣，诸如由360度相机、多角度相机阵列或光场相机生成的那些。由这些相机提供的沉浸式体验可以提高包括运动、训练（例如，建筑安全），和虚拟勘探（例如，大学步行道，历史遗迹）。利用来自这些摄像机的内容，用户不仅仅被动地消费内容，而是可以从他们选择的视角沿着沿着许多不同的路径交互地遍历内容，其中不同的用户观察相同内容的不同视角。要在互联网规模上支持这一点是具有挑战性的，客户端播放器必须能够切换视角与低延迟，视角可能需要生成的视频服务器的需求，和基础设施必须支持各种设备的捕获和消费的内容。这个项目探讨架构增强，算法和技术，以提供多视角视频在互联网规模。它将传输优化与视频编码和人机交互相结合。该项目将开发交互性抽象，内容发布者可以通过这些抽象来指定用户在视频中的每个点上可以选择的视角范围。视频中指定的交互性将（i）驱动视角编码和新颖的动态视角生成算法;（ii）使基础设施供应能够满足内容分发网络（CDN）存储和成本约束;以及（iii）将引导从CDN服务器或从附近的缓存进行自适应视角检索。最后，该项目将探索预测和指导用户行为的方法，以提高基于用户研究的交付质量。通过在互联网规模上为多视角视频提供更具响应性和可预测性的用户体验，并通过设计能够科普此类视频增长的网络，该项目将影响媒体，IT，教育和网络行业。工发组织将召集一个由相关从业人员组成的工业咨询委员会，并举办讲习班以促进合作。这项研究不仅将广泛涉及研究生，而且还将涉及来自代表性不足社区的本科生，将创建新的课程材料，以及一个新的本科生设计项目，涉及从大一到大四的学生，并有妇女和代表性不足的学生的强烈参与。该项目的结果将在以下网址提供： https://engineering.purdue.edu/~isl/multi-perspective-video.htmland https://nsl.usc.edu/projects/multi-perspective-video/The该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The success of streaming video has generated interest in newer forms of multi-perspective video content, such as those generated by 360-degree cameras, multi-angle camera arrays, or light-field cameras. The immersive experience provided by these cameras can enhance user satisfaction in domains including sports, training (e.g., construction safety), and virtual exploration (e.g., college walkthroughs, historical sites). With content from these cameras, users do not just passively consume content, but may interactively traverse the content along many different paths from a perspective of their choice, with different users observing different perspectives of the same content. To support this at Internet-scale is challenging; client players must be able to switch perspectives with low latency, the perspectives may need to be generated on demand by video servers, and the infrastructure must support a variety of devices for capture and consumption of this content.This project explores architectural enhancements, algorithms, and techniques to deliver multi-perspective video at Internet-scale. It couples delivery optimization with video coding and human computer interaction. The project will develop interactivity abstractions by which content publishers can specify the range of perspectives users are permitted to choose from at each point in the video. The interactivity specified in a video will (i) drive perspective coding and novel dynamic perspective generation algorithms; (ii) enable infrastructure provisioning to meet Content Delivery Network (CDN) storage and cost constraints; and (iii) will guide adaptive perspective retrieval from CDN servers or from nearby caches. Finally, the project will explore methods to predict and guide user behavior to improve delivery quality based on user studies. By enabling more responsive and predictable user experience for multi-perspective video at Internet scale, and by designing networks that can cope with growth in such video, the project will impact the media, IT, education, and networking industries. It will convene an Industrial Advisory Board comprising relevant practitioners and organize workshops to foster collaborations. The research will not only extensively involve graduate students, but also undergraduate students from underrepresented communities, will create new curriculum material, as well as a new undergraduate design project that involves students ranging from the freshman to senior year with strong participation from women and underrepresented students. The results from the project will be available at: https://engineering.purdue.edu/~isl/multi-perspective-video.htmland https://nsl.usc.edu/projects/multi-perspective-video/The results will be available throughout the duration of the project, and for three years after the project ends.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.