Using computer science to understand human perception, measure performance and redesign the video production pipeline.

点击翻译按钮获取中文摘要

The video production pipeline relies heavily on manual labour. Consequently, the process is vulnerable to issues (e.g., "quantity vs. quality") that often compromise the artistic integrity, audience perception and financial viability of the project. The pipeline also tends to resist innovation and creativity because of the cost of retraining the labour pool for new technologies. This impedes the ability of artists to produce innovative content using new media such as stereo 3D (S3D), high frame rate, and augmented reality. The proposed research will explore computer science innovations to find a better balance between manual labour and automation in the video production pipeline. I want to improve the productivity of the manual labour, which should focus on creativity instead of repetitive tasks that might be automated. In particular, I want to investigate if computer science can assist with or eliminate these manual and repetitive tasks. Most recent blockbuster films have been released in both 2D and S3D formats. The higher priced S3D tickets provide audiences with an experience they cannot get at home, ideally nurturing audience satisfaction and increasing ticket sales. However, most S3D films are planned, shot, and edited in 2D and then converted to S3D afterward. Unfortunately, cinematography that works in 2D does not always work in 3D--leaving the audience disappointed by visual discomfort. My research will investigate methods to avoid the mistakes that lead to this discomfort, building bridges between computer science, kinesiology, and psychology. Many traditional pre-production pipeline techniques are inadequate for S3D content. For example, while storyboarding--a cartoon-like representation of shots--captures visual language, position, and basic motion, it cannot capture actual depth, making it difficult or too expensive for filmmakers to plan and visualize how S3D will be used in their film. This results in costly mistakes (e.g., uncomfortable depth) that require significant manual labour to fix. My research will investigate new methods for planning and visualization of S3D content and correcting mistakes. Over the last one hundred years, standard cinematographic conventions have developed, often referred to collectively as film language. Film language theory could be used to identify and correct problems during early planning to prevent issues during filming. This is especially important for 3D, where filmmakers have not learned and practiced the guidelines that avoid problems. I propose to investigate the informal nature of film language to find a more rigorous definition, combining techniques from formal and natural language processing to create a programming language with autocorrection. While this can be used by filmmakers of any experience level, autocorrection would be of particular use to amateur filmmakers looking to learn and improve the quality of their work and could become part of consumer-level phones and cameras.