Cognitive Shadow: capturing judgement policies for real-time decision support

形式化地获取专家知识是决策支持领域的一大挑战。策略捕获是一种根据过去的例子来推断决策策略的判断分析方法。当一个模型的表现实际上超过了该模型所基于的人类决策者(通常是因为该模型不容易疲劳或分心)时，就可以观察到“判断自举”效应。开发了一个名为Cognitive Shadow的原型策略捕获工具，用于自动学习操作员的决策模式，并在检测到用户的决策与推断的决策策略之间不匹配时实时提供警告。该项目旨在研究在海军防空背景下实施认知影子能力的关键人为因素和认知工程问题。具体目标是1)评估影响操作员对认知助手的信任的因素，2)评估对工作负荷的影响， 行为和性能测量，以及3)了解两种可能的操作模式的不同影响：跟踪模式，其中操作员仅在检测到与决策模型不匹配时才接收警报，而推荐模式，操作员接收关于模型的预期决策的信息。为了系统地研究这些问题，将分两个阶段进行一系列人在环实验。在第一阶段中，模拟作战控制系统试验台被用作加拿大海军哈利法克斯级护卫舰上练习的海军防空的功能模拟。第一阶段将涉及两个实验室实验，涉及实施认知影子假设，以不同的方式影响工作量、信任、绩效和能力保持。阶段2将涉及在真实世界战斗管理系统的上下文中与领域专家一起使用认知阴影。该项目将帮助泰利斯研究和技术加拿大公司在成熟新的认知阴影方面取得飞跃 加拿大皇家海军的技术进步与加拿大国防研究与发展局的合作。

Formally capturing expert knowledge is a major challenge in the field of decision support. Policy capturing is a judgment analysis method that infers a decision policy based on past examples. A "judgmental bootstrapping" effect can be observed when a model actually outperforms the human decision maker on which the model was based (typically because the model is not prone to fatigue or distraction). A prototype policy capturing tool called the Cognitive Shadow has been developed to automatically learn an operator's decision pattern and provide warnings in real-time when a mismatch is detected between the user's decision and the inferred decision policy. This project aims at investigating critical human factors and cognitive engineering issues for implementing the Cognitive Shadow capability in the context of naval air defence. The specific objectives are to 1) assess factors influencing operator trust in the cognitive assistant, 2) assess impacts on workload, behavior and performance measures, and 3) understand the different impacts of two possible modes of operation: a shadowing mode where the operator receives an alert only when a mismatch with the decision model is detected, versus a recommendation mode where the operator receives information about the model's expected decision. A series of human-in-the-loop experiments will be performed in two phases in order to systematically investigate these issues. In Phase 1 the Simulated Combat Control System testbed is used as a functional simulation of naval air defence practiced aboard the Canadian Navy's Halifax class frigates. Phase 1 will involve two laboratory experiments involving implementations of the Cognitive Shadow hypothesized to differentially impact workload, trust, performance and competency preservation. Phase 2 will involve using the Cognitive Shadow with domain experts in the context of a real-world combat management system. This project will help Thales Research and Technology Canada take a leap forward in maturing the new Cognitive Shadow technology, and Defence Research and Development Canada in leveraging the technological advances for the Royal Canadian Navy.