Interaction Analytics for Automatic Assessment of Communication Quality in Primary Care

这个博士项目的目标是研究医生和病人之间的互动和沟通。研究已经证明了医生的人际交往能力对具体医疗结果的影响，无论是在与病人互动还是在团队内部。最近在论证和谈判建模领域的工作表明，自然语言处理（NLP）和机器学习可以用来监测和评估表面和认知沟通技巧。此外，同样的技术已开始应用于医疗领域，例如精神健康状况测试，以评估和诊断阿尔茨海默病和痴呆症。在本博士的范围内，正在研究两种方案的互补性。第一个场景是一对一的交流，即在全科医生办公室进行医疗访谈的医生和病人。在这种情况下，重点是根据最先进的做法和指导方针监测咨询的不同方面。除了对医生的监控外，还包括对病人话语和行为的分析。这一场景的参考是《剑桥医学访谈指南》的各个阶段：收集信息、提供结构、建立关系、解释和规划、跟踪信息和结束会话。第二种情况是医疗紧急情况下的多对多通信，例如重症监护病房。由于患者处于危重状态，将不直接跟踪，但将探讨以生命体征形式间接数据采集的贡献。医生和护士在危急和紧张情况下的干预措施涉及一套额外的、不同的技能。在这种情况下，重点放在危机资源管理（CRM）上，这是一个来自航空业并在航空业中使用的概念。这是围绕非技术技能（NTS），一套一般的认知和人际交往能力。NTS补充了领域技能，对于实现安全和有效的干预措施至关重要。这两种情况下的核心分析具有相似的属性，集中在计划、沟通、自信和信息利用方面。将根据不同的输入方式及其组合开发新的多模态解释模型。将同时开发一个研究平台，以包括和测试这些模型。第一部分视频和3D识别将用于姿势，手势和面部表情的解释。口译的第二部分将使用完整的自然语言理解（NLU）管道，包括基于自动语音识别和说话人日记的韵律特征分析和语义处理。话语建模，即对语境信息的高水平解释，将在融合低水平解释的基础上发展，以探索更复杂的模式和结构，并以传播绩效研究的结论和方向为基础。当前的工作是基于需求的定义和记录设置的规范，该规范基于对先前在域中执行的数据收集的分析。目标是为获取高质量的匿名数据集提供指南，这些数据集将有利于手动和自动分析，并适合高级处理。

The goal of this PhD project is to investigate the interaction and communication of between doctors and patients.Studies have demonstrated the effect of doctors' interpersonal skills on concrete medical outcomes, both while interacting with patients and within teams. Recent work in the domain of argumentation and negotiation modelling has shown that natural language processing (NLP) and machine learning can be used to monitor and assess surface and cognitive communication skills. In addition, the same techniques have started to be deployed in the medical domain, e.g. tests in mental health conditions screening for assessment and diagnosis of Alzheihmer's disease and dementia.In the scope of this PhD, two scenarios are being investigated for their complementarity. The first scenario features one-to-one communication, i.e. doctor-patient during medical interview in a GP's office setting. In this scenario, the focus is to monitor the different aspects of the consultation, based on state of the art practices and guidelines. In addition to the monitoring of the practitioner, it involves the analysis of the patient's discourse and behaviour. The reference for this scenario are the phases of the Cambridge guide to the medical interview: gathering information, providing structure, relationship building, explanation and planning, tracking information and closing the session. The second scenario features many-to-many communication in medical emergency settings, e.g. intensive care units. The patient, being in critical condition, will not be directly tracked but the contribution of indirect data acquisition in the form of vital signs will be explored. Doctors and nurses interventions in critical and stressful situations involve an additional, different set of skills. In this context, the focus is set on crisis resource management (CRM), a concept emerging from, and in use in the aviation industry. This is centred around non-technical skills (NTS), a set of general cognitive and interpersonal skills. NTS complements domain skills and is crucial to allow safe and efficient interventions.The core analysis in these two scenarios share similar attributes centred around planning, communication, assertiveness and utilisation of information. New models of multimodal interpretation will be developed based on the different inputs modalities and their combination. A research platform will be developed in parallel to include and test the models. The first part Video and 3D recognition will be used for posture, gesture and facial expressions interpretation. The second part of the interpretation will use a complete Natural Language Understanding (NLU) pipeline, including prosodic features analysis and semantic processing based on automatic speech recognition and speaker diarisation. Discourse modelling, i.e. high level interpretation of contextual information, will then be developed based on the fused low level interpretations for the exploration of more complex patterns and structures, grounded in conclusions and directions of studies in communication performance.The current work is set on the definition of the requirements and the specification of the recording setups based on the analysis of data collections previously performed in the domain. The objective is to offer guidelines for the acquisition of high quality anonymised dataset that would benefit both manual and automatic analysis and would be suitable for advanced processing.