HCC-Small: A Cognitive Assistive System for Coaching the Use of Home Medical Devices

该研究项目将开发方法和算法，以帮助人们操作家庭医疗设备所需的程序，而不会出现错误。目标是让系统在正确操作设备的人员的样本记录上进行训练，以便以后能够检测到当前执行过程中与正确操作顺序的偏差，并自动向用户提供纠正反馈，包括显示适当步骤的视频部分片段。该项目为通过观察学习提供了一种新的范例，它不需要完全理解任意视觉和传感器序列中的详细活动，而只需将已知环境中的给定新序列与先前建立的训练数据对齐，以检测显著偏差。该方法有四个组成部分：(1)定义操作程序中的关键状态以及最佳检测和随后通信便携式家用医疗设备的正确操作所需的传感器；(2)通过观察多个正确操作来训练系统；(3)观察操作序列的新实例，并认识到该操作与训练数据有显著偏差；(4)以音视频提示的形式向用户提供纠正反馈。本研究旨在了解家用医疗设备操作中常见的步骤类型，绘制如何通过适当的传感器检测这些步骤的关键指标，训练系统在特定操作人员的背景下识别这些步骤，建立不同操作步骤类型和相应传感器所需的训练重复范围，并在发生错误时为最终用户提供一套合适的干预措施。实验将为设备操作的自动分类建立训练数据集大小的范围。该研究期望从一组设备（如输液泵）的观察角度得出典型操作步骤的分类，并建立最有效的传感器或传感器组合来检测每种步骤的成功完成。此外，这项工作将有助于在训练观察的录像部分找到适当的段落，作为纠正反馈，以及可能适合特定用户的其他互动对话干预。本研究的长期目标是开发一种认知辅助系统，通过多传感器观察和与人类操作员的交互，学习和表示家庭医疗设备操作中的步骤序列。一些目标家庭医疗保健设备的例子是呼吸器和雾化器（帮助呼吸）、透析机、输液泵、家庭血氧脉搏监测设备、脑电图和心电图。该项目将为这些设备的家庭用户开发一种方法，以确保遵循正确的程序和准确的操作结果。该系统提供持续的反馈，以帮助用户在他们的设备操作通过观察？该过程通过不同的传感技术，并在需要时提供适当的指导。从这项工作中立即受益的目标人群将是患有轻度认知障碍的患者，他们将在使用家庭医疗设备时得到自动教练的支持。不断增长的用户群包括住在家里，但需要家用医疗设备支持的老年人。只要家用医疗设备提供所需的健康支持，这些医疗设备就可以使患者在最少的帮助下独立生活。最终结果可能是在操作和维护家用医疗设备方面减少错误和求助的次数。这将使人们在家中独立生活的平均时间比目前更长，从而降低医疗保健系统的成本。该研究对医疗设备公司在设备设计、验证和验证过程方面很有价值，提供了关于哪些传感器和通信设备最有利于集成到设备本身的见解。

This research project will develop methods and algorithms to assist people with the procedures required to operate their home medical devices without errors. The goal is for the system to be trained on sample recordings of the person operating the device correctly, so that later it will be able to detect deviations from the correct operation sequence in the currently performed procedure, and automatically provide corrective feedback to the user, including segments of the video portions that show the appropriate steps. The project provides a new paradigm for learning by observation that does not require complete understanding of detailed activities in arbitrary visual and sensor sequences, but merely aligns a given new sequence in known context with previously established training data to detect significant deviations. The approach has four components: (1) defining the key states in an operational procedure and the sensors required to best detect and later communicate the proper operation of a portable home medical device; (2) training the system by observing multiple correct operations; (3) observing a new instance of the operation sequence and recognizing that this operation deviates from the training data in a significant way; and (4) providing corrective feedback to the user in the form of audio and video prompts. The research aims to understand the common types of steps required in the operations of home medical devices, map how the critical indicators of these steps can be detected through appropriate sensors, train a system to recognize these steps in the context of a specific human operator, establish a range of required training repetitions for different operational step types and corresponding sensors, and provide a set of suitable interventions to the end user when errors occur. The experiments will establish the range of training data set sizes for the automated classification of device operations. The research expects to yield a taxonomy of typical operational steps from an observational perspective for a set of devices such as infusion pumps, and establish the most effective sensors or sensor combinations to detect the successful completion of each type of step. In addition, the work will help find suitable passages in the video portion of the training observations to use as corrective feedback, together with other interactive dialog interventions that may be appropriate for the particular user. The long-term goal of this research is to develop a cognitive assistance system to learn and represent sequences of steps in the operation of home medical devices through multi-sensor observation and interaction with a human operator. Examples of some of the targeted home healthcare devices are respirators and nebulizers (to help breathing), dialysis machines, infusion pumps, home monitoring devices for blood pulse oxygen, EEG, and ECG. The project will develop a means for home users of these devices to ensure that the correct procedures are followed and accurate operations result. The system provides ongoing feedback to assist users in their device operation by ?watching? the process via different sensing technologies and providing appropriate guidance when required. The target population immediately benefiting from this work would be patients with mild cognitive impairments who would be supported with the automated coach in their use of home medical devices. The growing user base includes elderly people living at home, but requiring support from home medical devices. These medical devices can allow a patient to live independently with minimal assistance, as long as the home medical devices provide the required health support. The end result may be a reduction in errors and in the number of calls for assistance in the operation and maintenance of home medical devices. This will allow people to live independently at home for an average longer period than at present and thereby reduce health care system costs. The research is valuable for medical device companies with respect to device design, verification, and validation processes, offering insights into what sensors and communication devices could be most beneficial for integration into the device itself.