Entrainment Analysis for Assessment of Autistic Speech Prosody Using Bottleneck Features of Deep Neural Network
Entrainment Analysis for Assessment of Autistic Speech Prosody Using Bottleneck Features of Deep Neural Network
复制标题
利用深度神经网络瓶颈特征评估自闭症言语韵律的夹带分析
DOI：
10.1109/icassp43922.2022.9746787
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yamasue Hidenori
中科院分区：
文献类型：
--
作者：
Ochi Keiko;Ono Nobutaka;Owada Keiho;Kuroda Miho;Sagayama Shigeki;Yamasue Hidenori
在本研究中，我们量化夹带的谈话的特点，目的是自动评估自闭症谱系障碍（ASD）的严重程度。我们关注的是话轮转换前后的话语对，它们具有韵律/声学相似性。ASD的临床严重程度是通过沙漏形深度神经网络（DNN）在用于测量夹带程度的神经夹带距离（NED）方法中获得的瓶颈特征来估计的。DNN首先在各种日常情况下使用大型对话语料库进行预训练，然后在自闭症诊断观察计划（ADOS）评估期间进行对话微调。从一对话语之间的瓶颈特征向量计算绝对差向量。将绝对差向量的质心和方差与我们先前研究中发现的语音特征相结合，以估计ASD严重程度的分数，因此，估计的分数与实际观察到的ADOS 'Reciprocity'分数显著相关，相关系数为0.70。这一结果表明，有效地使用微调技术与数据的典型发达国家的个人，此外，揭示了社会沟通缺陷的ASD个人所代表的话语相邻的话轮。
In the present study, we quantify entrainment characteristics of conversation with the aim of automatic assessment of the severity of autism spectrum disorder (ASD). We focus on pairs of utterances immediately before and after turn-takings, which have prosodic/acoustic similarities.The clinical severity of ASD is estimated by the bottleneck features obtained by an hourglass-shaped deep neural network (DNN) in the neural entrainment distance (NED) method used to measure the degree of entrainment. The DNN is firstly pre-trained using a large conversation corpus in various daily situations and then fine-tuned with conversations during the Autism Diagnostic Observation Schedule (ADOS) assessment. Absolute difference vectors are calculated from the bottleneck feature vectors between a pair of utterances. Centroid and variance of the absolute difference vectors are combined with the speech features discovered in our previous study in order to estimate the scores of ASD severity.Consequently, the estimated scores significantly correlate with the actual observed ADOS ‘Reciprocity' scores with a coefficient of 0.70. This result shows the effective use of finetuning technique with data of typically developed individuals and, furthermore, reveals social communication deficits in ASD individuals represented by utterances adjacent to turn-takings.