A Business / Market Opportunity Assessment for A Speech-Driven Facial Animation System

对于3D面部动画和语音同步的高度竞争的研究领域，电视/电影、角色驱动的视频游戏、AR/VR和移动的技术等应用越来越多地朝着跨所有平台的更沉浸式体验发展。这一转变给动画师和内容创作者带来了压力，他们需要在生产流程和方法上进行创新，以便在可能的情况下实现动画自动化，并采用新技术，使他们能够实时实现动画效果，但具有离线手工动画质量。我们提出了在这个建议，一个新的语音驱动的工作流程，表现力，动画师为中心的面部动画。我们表明，一个完整的，逼真的人脸性能可以自动生成的输入音频流和标记的文本成绩单，从解剖学，感知和心理语言学文献的见解。散布有方向标签的语音是动画师控制面部运动事件（眨眼、微笑或说话）时序的主要界面。动画师使用文字记录中的标签来注释角色面部在声乐表演中的情感、解剖学和非语言行为。对音频信号的分析通过将标记的转录与表演对齐来收集语音计时，并提取关键的音频线索来调用运动事件并调节情绪强度、讲话风格、眨眼和眉毛运动。分析后的输出用于按程序生成完整的动画面部，生成一组紧凑且稀疏的动画师可编辑的运动曲线，这些曲线可以交互式地驱动任何典型的基于FACS的面部装备。我们的工作流程已经成功地用八种语言为即将到来的AAA游戏制作了几个小时的动画，几乎不需要动画师后期编辑。这是一份I2 I市场评估资助申请，旨在接触现场和潜在客户，以确定理想的目标市场和痛点，从而为我们的创新技术成功商业化提供信息和阐明价值主张和市场进入战略。

For the highly competitive research field of 3D facial animation and speech synchronization, applications such as tv/film, character driven video games, AR/VR and mobile technologies are increasingly moving towards more immersive experiences, across all platforms. This shift places pressure on animators and content creators to innovate with their production pipelines and methodologies to automate animation where possible and to adopt new technologies that allow them to achieve animation results in real-time but with offline hand animated quality. We present in this proposal, a novel speech-driven workflow for expressive, animator-centric facial animation. We show that a complete, realistic human face performance can be automatically generated from an input audio stream and a tagged text transcript, using insights from anatomy, perception, and psycho-linguistic literature. Speech interspersed with directorial tags is the primary interface for animator control over the chronology of facial motion events (blinks, a smile, or an utterance). Animators use the tags in a transcript to annotate the emotional, anatomical, and non-verbal behaviour of a character's face during a vocal performance. An analysis of the audio signal gleans phonetic timing by aligning the tagged transcript with the performance, and it extracts key audio cues to invoke motion events and modulate emotional intensity, speech style, blinks and brow motion. The analyzed output is used to procedurally generate a complete animated face, producing a compact and sparse set of animator-editable motion curves that can interactively drive any typical FACS-based face rig. Our workflow has been used to successfully animate several hours of an upcoming AAA game in eight languages, with little to no need for animator post-editing. This is an application for an I2I Market Assessment grant to reach out to the field and potential customers in order to identify ideal target markets and pain points that will inform and articulate the value proposition and market entry strategy for the successful commercialization of our innovative technology.