A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System

对于竞争激烈的3D面部动画和语音同步研究领域，电视/电影、角色驱动视频游戏、AR/VR和移动技术等应用程序正越来越多地跨所有平台走向更身临其境的体验。这一转变给动画师和内容创作者带来了压力，他们必须创新自己的制作管道和方法，尽可能地实现动画的自动化，并采用新技术，使他们能够以离线的手工动画质量实现实时的动画效果。在这项提案中，我们提出了一种新颖的语音驱动的工作流程，用于以动画师为中心的表现力面部动画。我们表明，使用来自解剖学、知觉和心理语言文献的见解，可以从输入的音频流和标记的文本转录中自动生成完整的、真实的人脸表演。穿插着导演标签的语音是动画师控制面部运动事件(眨眼、微笑或说话)时间顺序的主要界面。动画师使用文字记录中的标签来注释角色在声乐表演中面部的情感、解剖和非语言行为。对音频信号的分析通过将标记的文字记录与表演对齐来收集语音计时，并提取关键音频线索来调用动作事件，并调节情绪强度、语音风格、眨眼和眉毛运动。分析的输出用于以程序方式生成完整的动画面，从而生成一组紧凑且稀疏的可由动画师编辑的运动曲线，这些曲线可以交互地驱动任何典型的基于FACS的面部装备。我们的工作流程已经成功地用八种语言为即将到来的AAA游戏制作了几个小时的动画，几乎不需要动画师的后期编辑。这是一份申请I2I市场评估基金的申请，目的是接触现场和潜在客户，以确定理想的目标市场和痛点，这些市场和痛点将为我们的创新技术成功商业化提供信息并阐明价值主张和市场进入战略。

For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.