Interaction Analytics for Automatic Assessment of Communication Quality in Primary Care

这个博士项目的目标是调查医生和患者之间的互动和沟通。研究表明，无论是在与患者互动时，还是在团队中，医生的人际交往技能对具体的医疗结果都有影响。最近在辩论和谈判建模领域的研究表明，自然语言处理(NLP)和机器学习可以用来监测和评估表面和认知沟通技能。此外，同样的技术已开始应用于医学领域，例如，精神健康状况检测，以评估和诊断阿尔茨海默氏病和痴呆。在本博士学位的范围内，正在调查两种情况的互补性。第一个场景以一对一交流为特征，即在全科医生的办公室环境中进行医生和患者的医学访谈。在这种情况下，重点是根据最先进的做法和指导方针，监测咨询的不同方面。除了对医生的监督之外，它还包括对病人的话语和行为的分析。这种情况的参考是剑桥医学访谈指南的几个阶段：收集信息、提供结构、建立关系、解释和规划、跟踪信息和结束会议。第二种情况的特点是医疗急救环境中的多对多通信，例如重症监护病房。处于危急状态的患者将不会被直接跟踪，但将探索以生命体征的形式进行间接数据采集的贡献。医生和护士在危急和有压力的情况下的干预涉及一套额外的、不同的技能。在这方面，重点放在危机资源管理(CRM)上，这是一个从航空业产生并在航空业中使用的概念。这是以非技术技能(NTS)为中心的，这是一套一般的认知和人际交往技能。NTS是对领域技能的补充，对于实现安全和有效的干预至关重要。这两个情景中的核心分析以规划、沟通、自信和信息利用为中心，具有相似的属性。将根据不同的输入模式及其组合开发新的多模式口译模式。将并行开发一个研究平台，以纳入和测试这些模型。第一部分视频和3D识别将用于姿势、手势和面部表情的解读。口译的第二部分将使用完整的自然语言理解(NLU)管道，包括基于自动语音识别和说话人日记的韵律特征分析和语义处理。语篇建模，即语境信息的高层次解释，将在融合的低层次解释的基础上开发，以探索更复杂的模式和结构，其基础是交际性能研究的结论和方向。目前的工作是基于对先前在该领域进行的数据收集的分析，确定需求和录音装置的规范。其目的是为获取高质量的匿名数据集提供指导方针，这些数据集将有利于手动和自动分析，并将适用于高级处理。

The goal of this PhD project is to investigate the interaction and communication of between doctors and patients.Studies have demonstrated the effect of doctors' interpersonal skills on concrete medical outcomes, both while interacting with patients and within teams. Recent work in the domain of argumentation and negotiation modelling has shown that natural language processing (NLP) and machine learning can be used to monitor and assess surface and cognitive communication skills. In addition, the same techniques have started to be deployed in the medical domain, e.g. tests in mental health conditions screening for assessment and diagnosis of Alzheihmer's disease and dementia.In the scope of this PhD, two scenarios are being investigated for their complementarity. The first scenario features one-to-one communication, i.e. doctor-patient during medical interview in a GP's office setting. In this scenario, the focus is to monitor the different aspects of the consultation, based on state of the art practices and guidelines. In addition to the monitoring of the practitioner, it involves the analysis of the patient's discourse and behaviour. The reference for this scenario are the phases of the Cambridge guide to the medical interview: gathering information, providing structure, relationship building, explanation and planning, tracking information and closing the session. The second scenario features many-to-many communication in medical emergency settings, e.g. intensive care units. The patient, being in critical condition, will not be directly tracked but the contribution of indirect data acquisition in the form of vital signs will be explored. Doctors and nurses interventions in critical and stressful situations involve an additional, different set of skills. In this context, the focus is set on crisis resource management (CRM), a concept emerging from, and in use in the aviation industry. This is centred around non-technical skills (NTS), a set of general cognitive and interpersonal skills. NTS complements domain skills and is crucial to allow safe and efficient interventions.The core analysis in these two scenarios share similar attributes centred around planning, communication, assertiveness and utilisation of information. New models of multimodal interpretation will be developed based on the different inputs modalities and their combination. A research platform will be developed in parallel to include and test the models. The first part Video and 3D recognition will be used for posture, gesture and facial expressions interpretation. The second part of the interpretation will use a complete Natural Language Understanding (NLU) pipeline, including prosodic features analysis and semantic processing based on automatic speech recognition and speaker diarisation. Discourse modelling, i.e. high level interpretation of contextual information, will then be developed based on the fused low level interpretations for the exploration of more complex patterns and structures, grounded in conclusions and directions of studies in communication performance.The current work is set on the definition of the requirements and the specification of the recording setups based on the analysis of data collections previously performed in the domain. The objective is to offer guidelines for the acquisition of high quality anonymised dataset that would benefit both manual and automatic analysis and would be suitable for advanced processing.