ExpressGesture: Expressive gesture generation from speech through database matching
ExpressGesture: Expressive gesture generation from speech through database matching
复制标题
ExpressGesture：通过数据库匹配从语音生成富有表现力的手势
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
R. Mcdonnell
中科院分区：
文献类型：
--
作者：
Ylva Ferstl;Michael Neff;R. Mcdonnell
协同语音手势是使虚拟代理更加人性化和吸引人的重要因素。基于语音输入自动生成的手势通常缺乏真实和定义的手势形式。我们提出了一个数据库驱动的方法，保证定义的手势形式。我们建立了一个包含超过23，000个动作捕捉的共同语音手势的大型语料库，并根据可以从语音音频中估计的表达性手势特征选择单个手势。表达参数是手势速度和加速度、手势大小、手臂旋转和手指伸展。然后将各个参数匹配的手势组合成动画序列。我们评估我们的手势生成系统在两个感知研究。第一项研究将我们的方法与地面实况手势以及不匹配的手势进行了比较。第二项研究将我们的方法与当前五种生成式机器学习模型进行了比较。我们的方法在第一项研究中优于不匹配的手势选择，并在第二项研究中表现出竞争力。
Co‐speech gestures are a vital ingredient in making virtual agents more human‐like and engaging. Automatically generated gestures based on speech‐input often lack realistic and defined gesture form. We present a database‐driven approach guaranteeing defined gesture form. We built a large corpus of over 23,000 motion‐captured co‐speech gestures and select individual gestures based on expressive gesture characteristics that can be estimated from speech audio. The expressive parameters are gesture velocity and acceleration, gesture size, arm swivel, and finger extension. Individual, parameter‐matched gestures are then combined into animated sequences. We evaluate our gesture generation system in two perceptual studies. The first study compares our method to the ground truth gestures as well as mismatched gestures. The second study compares our method to five current generative machine learning models. Our method outperformed mismatched gesture selection in the first study and showed competitive performance in the second.