CHS: Small: Extracting affect and interaction information from primary care visits to support patient-provider interactions

初级保健是强调持续和预防性保健的有效保健系统的一个组成部分。然而，美国面临着初级保健医生短缺的问题，预计这种情况将变得更糟，在为农村和少数民族患者服务的诊所中，这种短缺最为明显。这种短缺可能导致压力和医生倦怠，影响短缺本身和患者-提供者互动的质量。该项目将开发用于初级保健访问的半自动分析方法，其长期目标是检测瞬时和长期压力，然后提供反馈和反映系统，以帮助减轻压力，提高患者护理质量和医生的保留率。这将需要在视频处理方面的进步，以认识到人类如何相互作用和技术，这些相互作用如何影响群体的情感状态和关系的新模型，以及在初级保健访问期间对患者和提供者需求的初步设计调查。这反过来又需要为卫生服务和工程专业的学生提供跨学科的教育机会。该项目有两个主要目标。第一个重点是开发方法，从临床遇到的压力，倦怠和互动中提取数据，利用大量现有的数据集记录的初级保健访问收集在几个方面。该团队将开发技术来识别互动中的非语言和语言线索，包括眼神接触，面部表情，姿势和肢体语言，轮流行为以及社会情感交流的指标，如面部模仿。该团队还将通过凝视分析、打字或触摸屏使用的证据以及屏幕共享/共同引用行为来提取有关参与者如何与技术交互的线索。这些方法将通过将其性能与人类注释进行比较来进行验证，提取的数据将为第二个主要目标提供信息，即确定开发工具以提供反馈和支持反思的机会和要求。第二个目标将通过以用户为中心的设计方法来实现，包括与临床合作者一起审查提取的数据以评估其潜在价值，通过人种学观察方法对患者-提供者互动进行工作流程分析，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准。

Primary care is an integral part of an effective health care system that emphasizes continuous and preventive care. However, the United States faces a shortage of primary care physicians that is projected to get worse, a shortage most pronounced in clinics that serve rural and minority patients. This shortage can lead to stress and physician burnout, affecting both the shortage itself and the quality of patient-provider interactions. This project will develop methods for semi-automated analysis of primary care visits, with the long-term goal of detecting both momentary and longer-term stress, then providing feedback and reflection systems to help reduce stress and improve the quality of patient care and retention of physicians. This will require both advances in video processing to recognize aspects of how humans interact with both each other and with technology, novel models of how these interactions affect the group's affective state and relationship, and initial design inquiry into patients' and providers' needs during primary care visits. This, in turn, will require providing interdisciplinary educational opportunities for health service and engineering students. The project has two main aims. The first focuses on developing methods to extract data from clinical encounters about stress, burnout, and interaction, leveraging a large existing dataset of recorded primary care visits collected in several contexts. The team will develop techniques to recognize both non-verbal and verbal cues in interactions, including eye contact, facial expressions, posture and body language, turn-taking behaviors, and indicators of socio-emotional exchange such as facial mimicry. The team will also extract cues about how the people involved interact with technology through gaze analysis, evidence of typing or touchscreen use, and screen sharing/co-referencing behaviors. These methods will be validated by comparing their performance against human annotations, with the extracted data informing the second main aim around determining opportunities and requirements for developing tools to provide feedback and support reflection. This second aim will be pursued through user-centered design methods, including reviewing extracted data with clinical collaborators to evaluate its potential value, workflow analysis of patient-provider interactions informed by ethnographic observational methods, and thematic analysis of focus groups including both patients and care providers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.