Facial Expression Analysis by Image Processing

描述(由申请人提供)：面部表情提供情绪反应的线索，调节人际行为，并传达精神病理学的各个方面。基于人-观察者的面部表情测量方法是劳动密集型、定性的、很难标准化的。我们由计算机和行为科学家组成的跨学科团队开发了CMU/PIT自动面部图像分析(AFA)系统，该系统能够自动识别面部动作单位并分析它们在面部行为中的时机。与人工和主观测量相比，AFA实现的定量测量是一个重大进步，不需要使用侵入性传感器。我们设想使用AFA对情绪表达和相关非语言行为的可靠、有效和有效的测量来评估抑郁症的症状严重程度。目前对抑郁症的临床评估方法几乎完全依赖于口头报告(临床访谈和/或问卷)。它们缺乏系统和有效的方法来整合行为观察，而行为观察是抑郁症状的强有力指标，特别是那些与临床医生和患者之间的二元互动时间相关的行为观察，其中大部分可能发生在两个人都不知道的情况下。AFA能够提取抑郁症的非语言指标的类型和时间。我们的假设是，通过AFA获得的面部表情、头部运动和凝视的结构和时机的量化测量，当结合来自访谈和自我报告问卷的信息时，将改善对症状严重程度的临床评估和对治疗结果的评估。我们建议在40名参与重度抑郁症治疗干预研究的参与者中检验这一假说。访谈、问卷和视频数据将在治疗过程中定期收集。为了测量社会动态，患者和面试者都将使用AFA进行视频记录和处理。纵向多水平模型将被用来检验研究假设。我们将进一步改进AFA的算法和能力，以满足评估目标，并准备AFA供科学和临床使用。

DESCRIPTION (provided by applicant): Facial expression provides cues about emotional response, regulates interpersonal behavior, and communicates aspects of psychopathology. Human-observer based methods for measuring facial expression are labor intensive, qualitative, and difficult to standardize. Our interdisciplinary team of computer and behavioral scientists has developed the CMU/Pitt Automated Facial Image Analysis (AFA) system that is capable of automatically recognizing facial action units and analyzing their timing in facial behavior. The quantitative measurement achieved by AFA represents a major advance over manual and subjective measurement without requiring the use of invasive sensors. We envision to use AFA's reliable, valid, and efficient measurement of emotion expression and related nonverbal behavior for assessment of symptom severity in depression. Current methods of clinical assessment of depression depend almost entirely on verbal report (clinical interview and/or questionnaire). They lack systematic and efficient ways of incorporating behavioral observations that are .strong indicators of depressive symptoms, especially those related to the timing of dyadic interaction between clinician and patient, much of which may occur outside the awareness of either individual. AFA is capable of extracting both the type and timing of nonverbal indicators of depression. Our hypothesis is that quantitative measures of the configuration and timing of facial expression, head motion, and gaze obtainable by AFA will improve clinical assessment of symptom severity and evaluation of treatment outcomes when combined with information from interviews and self-report questionnaires. We propose to test this hypothesis in 40 participants participating in a treatment intervention study for major depression. Interview, questionnaire, and video data will be collected at regular intervals over the course of treatment. To measure social dynamics, both patient and interviewer will be video recorded and processed using AFA. Longitudinal multilevel modeling will be used to test study hypotheses. We will improve further algorithms and capabilities of AFA to meet evaluation goals and prepare AFA for use by the scientific and clinical community.