A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System

点击翻译按钮获取中文摘要

For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.