A belief-based sequential fusion approach for fusing manual signs and non-manual signals
A belief-based sequential fusion approach for fusing manual signs and non-manual signals
复制标题
DOI：
10.1016/j.patcog.2008.09.010
复制
发表时间：
2009-05
期刊：
影响因子：
--
通讯作者：
O. Aran;Thomas Burger;A. Caplier;L. Akarun
中科院分区：
文献类型：
--
作者：
O. Aran;Thomas Burger;A. Caplier;L. Akarun
目前对手语识别的研究大多集中在对手势(手势和形状)的识别上，而忽略了手语识别中一个非常重要的组成部分：非手语信号(面部表情和头肩运动)。我们使用一种顺序的基于信念的融合技术来识别包含人工和非人工成分的符号。在第一阶段中利用携带首要信息的手动组件。第二阶段使用非人工部件，只有在第一阶段的决策中有犹豫时才使用。我们使用信念形式主义来模拟犹豫不决，并确定在第二阶段中发生歧视的符号簇。我们已经在Sign Tutor应用程序中实现了这项技术。我们在eNTERFACE‘06 ASL数据库上的结果表明，与使用手动和非手动特征并行或特征融合的基线系统相比，我们获得了81.6%的准确率。
Most of the research on sign language recognition concentrates on recognizing only manual signs (hand gestures and shapes), discarding a very important component: the non-manual signals (facial expressions and head/shoulder motion). We address the recognition of signs with both manual and non-manual components using a sequential belief-based fusion technique. The manual components, which carry information of primary importance, are utilized in the first stage. The second stage, which makes use of non-manual components, is only employed if there is hesitation in the decision of the first stage. We employ belief formalism both to model the hesitation and to determine the sign clusters within which the discrimination takes place in the second stage. We have implemented this technique in a sign tutor application. Our results on the eNTERFACE’06 ASL database show an improvement over the baseline system which uses parallel or feature fusion of manual and non-manual features: we achieve an accuracy of 81.6%.