Using computer science to understand human perception, measure performance and redesign the video production pipeline.

利用计算机科学来理解人类感知、衡量性能并重新设计视频制作流程。

• 批准号: RGPIN-2019-04072
• 负责人: Istead, Lesley
• 金额: $ 1.39万
• 依托单位: Carleton University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=737386
• 关键词: Using; computer; science; understand; human

The video production pipeline relies heavily on manual labour. Consequently, the process is vulnerable to issues (e.g., "quantity vs. quality") that often compromise the artistic integrity, audience perception and financial viability of the project. The pipeline also tends to resist innovation and creativity because of the cost of retraining the labour pool for new technologies. This impedes the ability of artists to produce innovative content using new media such as stereo 3D (S3D), high frame rate, and augmented reality. The proposed research will explore computer science innovations to find a better balance between manual labour and automation in the video production pipeline. I want to improve the productivity of the manual labour, which should focus on creativity instead of repetitive tasks that might be automated. In particular, I want to investigate if computer science can assist with or eliminate these manual and repetitive tasks. Most recent blockbuster films have been released in both 2D and S3D formats. The higher priced S3D tickets provide audiences with an experience they cannot get at home, ideally nurturing audience satisfaction and increasing ticket sales. However, most S3D films are planned, shot, and edited in 2D and then converted to S3D afterward. Unfortunately, cinematography that works in 2D does not always work in 3D--leaving the audience disappointed by visual discomfort. My research will investigate methods to avoid the mistakes that lead to this discomfort, building bridges between computer science, kinesiology, and psychology. Many traditional pre-production pipeline techniques are inadequate for S3D content. For example, while storyboarding--a cartoon-like representation of shots--captures visual language, position, and basic motion, it cannot capture actual depth, making it difficult or too expensive for filmmakers to plan and visualize how S3D will be used in their film. This results in costly mistakes (e.g., uncomfortable depth) that require significant manual labour to fix. My research will investigate new methods for planning and visualization of S3D content and correcting mistakes. Over the last one hundred years, standard cinematographic conventions have developed, often referred to collectively as film language. Film language theory could be used to identify and correct problems during early planning to prevent issues during filming. This is especially important for 3D, where filmmakers have not learned and practiced the guidelines that avoid problems. I propose to investigate the informal nature of film language to find a more rigorous definition, combining techniques from formal and natural language processing to create a programming language with autocorrection. While this can be used by filmmakers of any experience level, autocorrection would be of particular use to amateur filmmakers looking to learn and improve the quality of their work and could become part of consumer-level phones and cameras.

视频制作流程严重依赖体力劳动。因此，该过程容易受到问题的影响（例如，“数量与质量”），往往损害艺术完整性，观众的看法和项目的财务可行性。由于对劳动力进行新技术再培训的成本，管道也往往抵制创新和创造力。这阻碍了艺术家使用新媒体（如立体3D（S3 D），高帧速率和增强现实）制作创新内容的能力。 拟议的研究将探索计算机科学创新，以在视频制作管道中找到手工劳动和自动化之间的更好平衡。我想提高体力劳动的生产率，这应该集中在创造力上，而不是可能被自动化的重复性任务。特别是，我想研究计算机科学是否可以帮助或消除这些手动和重复性的任务。 大多数最近的大片都以2D和S3 D格式发行。价格较高的S3 D门票为观众提供了他们在家里无法获得的体验，理想地培养观众满意度并增加门票销售。然而，大多数S3 D电影都是在2D中计划，拍摄和编辑，然后转换为S3 D。不幸的是，在2D中工作的电影摄影并不总是在3D中工作-让观众对视觉不适感到失望。我的研究将探讨避免导致这种不适的错误的方法，在计算机科学，人体运动学和心理学之间建立桥梁。许多传统的预生产流水线技术对于S3 D内容来说是不够的。例如，虽然故事板-一种类似卡通的镜头表示-捕捉视觉语言，位置和基本运动，但它无法捕捉实际深度，这使得电影制作者难以或过于昂贵地计划和可视化如何在电影中使用S3 D。这会导致代价高昂的错误（例如，不舒服深度），需要大量的手工劳动来固定。我的研究将探讨规划和可视化S3 D内容和纠正错误的新方法。在过去的一百年里，标准的电影惯例已经发展起来，通常统称为电影语言。电影语言理论可以用来识别和纠正早期计划中的问题，以防止拍摄过程中出现问题。这对于3D来说尤其重要，因为电影制作人还没有学习和实践避免问题的指导方针。我建议研究电影语言的非正式性质，以找到一个更严格的定义，结合正式和自然语言处理技术，创建一个具有自动校正功能的编程语言。虽然这可以被任何经验水平的电影制作人使用，但自动校正对于希望学习和提高工作质量的业余电影制作人来说特别有用，并可能成为消费级手机和相机的一部分。