A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System

在竞争激烈的3D面部动画和语音同步研究领域，电视/电影、角色驱动的视频游戏、AR/VR和移动技术等应用正越来越多地朝着跨所有平台的沉浸式体验发展。这种转变给动画师和内容创作者带来了压力，他们需要在生产流程和方法上进行创新，在可能的情况下实现动画的自动化，并采用新技术，使他们能够实时实现动画效果，但同时又具有离线手工动画的质量。我们提出了一种新颖的语音驱动工作流程，用于表达以动画师为中心的面部动画。我们展示了一个完整的、真实的人脸表演可以从输入的音频流和标记的文本文本中自动生成，使用解剖学、知觉和心理语言学文献的见解。穿插着指示标签的语音是动画师控制面部运动事件（眨眼、微笑或说话）的主要界面。动画师使用文本中的标签来注释角色在声乐表演期间的情感、解剖和非语言行为。音频信号的分析通过将标记的文本与表演相匹配来收集语音时间，并提取关键音频线索来调用运动事件并调节情绪强度、说话风格、眨眼和眉毛运动。分析的输出用于程序生成完整的动画人脸，生成一组紧凑且稀疏的动画师可编辑运动曲线，可以交互式地驱动任何典型的基于facs的人脸钻机。我们的工作流程已经成功地用8种语言制作了一款即将问世的AAA级游戏的几个小时动画，几乎不需要动画师的后期编辑。这是一份I2I市场评估补助金的申请，旨在接触现场和潜在客户，以确定理想的目标市场和痛点，从而为我们的创新技术的成功商业化提供信息和阐明价值主张和市场进入策略。

For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.