HCC-Small: A Cognitive Assistive System for Coaching the Use of Home Medical Devices

这项研究项目将开发方法和算法，以帮助人们按照所需的程序操作他们的家庭医疗设备，而不会出错。其目标是对该系统进行关于正确操作该设备的人员的样本记录的培训，以便稍后它将能够在当前执行的过程中检测与正确操作顺序的偏差，并自动向用户提供校正反馈，包括显示适当步骤的视频部分的片段。该项目提供了一种通过观察进行学习的新范式，它不需要完全了解任意视觉和传感器序列中的详细活动，而只需将已知环境中的给定新序列与先前建立的训练数据进行比对，以检测重大偏差。该方法具有四个组成部分：(1)定义操作过程中的关键状态以及最好地检测并随后传送便携式家庭医疗设备的正确操作所需的传感器；(2)通过观察多个正确的操作来训练系统；(3)观察操作序列的新实例并认识到该操作显著偏离训练数据；以及(4)以音频和视频提示的形式向用户提供校正反馈。该研究旨在了解家庭医疗设备操作中需要的常见步骤类型，绘制如何通过适当的传感器检测这些步骤的关键指标，训练系统在特定人类操作员的背景下识别这些步骤，为不同的操作步骤类型和相应的传感器建立一系列必需的重复训练，并在发生错误时向最终用户提供一套合适的干预措施。这些实验将为设备操作的自动分类建立训练数据集大小的范围。这项研究希望从观察的角度对输液泵等一系列设备的典型操作步骤进行分类，并建立最有效的传感器或传感器组合来检测每种步骤的成功完成。此外，这项工作将帮助在培训观察的视频部分找到合适的段落，用作纠正反馈，以及可能适合特定用户的其他交互式对话干预。这项研究的长期目标是开发一种认知辅助系统，通过多传感器观察和与操作员的交互来学习和表示家庭医疗设备操作的步骤序列。一些针对家庭保健设备的例子包括呼吸器和雾化器(帮助呼吸)、透析机、输液泵、家庭脉搏氧气监测设备、脑电和心电。该项目将为这些设备的家庭用户开发一种方法，以确保遵循正确的程序和准确的操作结果。该系统提供持续的反馈，通过观看帮助用户操作设备。这一过程通过不同的传感技术，并在需要时提供适当的指导。这项工作直接受益的目标人群将是轻度认知障碍的患者，他们将在使用家用医疗设备时得到自动化教练的支持。不断增长的用户群包括住在家里但需要家庭医疗设备支持的老年人。这些医疗设备可以让患者在最小限度的帮助下独立生活，只要家庭医疗设备提供所需的健康支持。最终的结果可能是减少家庭医疗设备的操作和维护中的错误和求助电话的数量。这将使人们能够在家中独立生活的平均时间比目前更长，从而降低医疗保健系统的成本。这项研究对医疗设备公司在设备设计、验证和验证过程方面具有价值，为哪些传感器和通信设备最有利于集成到设备本身提供了见解。

This research project will develop methods and algorithms to assist people with the procedures required to operate their home medical devices without errors. The goal is for the system to be trained on sample recordings of the person operating the device correctly, so that later it will be able to detect deviations from the correct operation sequence in the currently performed procedure, and automatically provide corrective feedback to the user, including segments of the video portions that show the appropriate steps. The project provides a new paradigm for learning by observation that does not require complete understanding of detailed activities in arbitrary visual and sensor sequences, but merely aligns a given new sequence in known context with previously established training data to detect significant deviations. The approach has four components: (1) defining the key states in an operational procedure and the sensors required to best detect and later communicate the proper operation of a portable home medical device; (2) training the system by observing multiple correct operations; (3) observing a new instance of the operation sequence and recognizing that this operation deviates from the training data in a significant way; and (4) providing corrective feedback to the user in the form of audio and video prompts. The research aims to understand the common types of steps required in the operations of home medical devices, map how the critical indicators of these steps can be detected through appropriate sensors, train a system to recognize these steps in the context of a specific human operator, establish a range of required training repetitions for different operational step types and corresponding sensors, and provide a set of suitable interventions to the end user when errors occur. The experiments will establish the range of training data set sizes for the automated classification of device operations. The research expects to yield a taxonomy of typical operational steps from an observational perspective for a set of devices such as infusion pumps, and establish the most effective sensors or sensor combinations to detect the successful completion of each type of step. In addition, the work will help find suitable passages in the video portion of the training observations to use as corrective feedback, together with other interactive dialog interventions that may be appropriate for the particular user. The long-term goal of this research is to develop a cognitive assistance system to learn and represent sequences of steps in the operation of home medical devices through multi-sensor observation and interaction with a human operator. Examples of some of the targeted home healthcare devices are respirators and nebulizers (to help breathing), dialysis machines, infusion pumps, home monitoring devices for blood pulse oxygen, EEG, and ECG. The project will develop a means for home users of these devices to ensure that the correct procedures are followed and accurate operations result. The system provides ongoing feedback to assist users in their device operation by ?watching? the process via different sensing technologies and providing appropriate guidance when required. The target population immediately benefiting from this work would be patients with mild cognitive impairments who would be supported with the automated coach in their use of home medical devices. The growing user base includes elderly people living at home, but requiring support from home medical devices. These medical devices can allow a patient to live independently with minimal assistance, as long as the home medical devices provide the required health support. The end result may be a reduction in errors and in the number of calls for assistance in the operation and maintenance of home medical devices. This will allow people to live independently at home for an average longer period than at present and thereby reduce health care system costs. The research is valuable for medical device companies with respect to device design, verification, and validation processes, offering insights into what sensors and communication devices could be most beneficial for integration into the device itself.