Prevalence effects in visual research: Theoretical and practical implications

视觉研究中的流行效应：理论和实践意义

• 批准号: 10362604
• 负责人: Jeremy M Wolfe
• 金额: $ 43.41万
• 依托单位: BRIGHAM AND WOMEN'S HOSPITAL
• 依托单位国家: 美国
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-04-01 至 2024-02-29
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10362604
• 关键词: Artificial Intelligence; Basic Science; Breast Cancer Detection; Cervical Cancer Screening; Collaborations; Cytology; Data; Detection; Effectiveness; Failure; Flecks; Goals; Human; Hybrids; Incidental Findings; Intervention; Joints; Low Prevalence; Malignant Neoplasms; Malignant neoplasm of lung; Methods; Modeling; Paper; Pattern; Performance; Pneumonia; Predictive Value; Prevalence; Prevalence Study; Reporting; Research; Research Personnel; Role; Screening for cancer; Security; Talents; Testing; Time; Training; Trust; Visual; Work; analog; base; clinically significant; deep learning; design; improved; mental representation; programs; side effect; social; theories; tool; vigilance; visual search

Low prevalence searches form an important and problematic class of visual search tasks. These are tasks where the search target is rare. Many socially important tasks like airport security or cancer screening are low prevalence tasks. Previous work, much of it from our lab, has shown that low prevalence can have undesirable effects. Most notably, miss (false negative) errors are markedly elevated at low prevalence. This is a clear problem if the purpose of the search is to detect something rare but important like cancer or a terrorist threat. Our previous work has documented this pattern of increased miss errors in a number of expert domains including cytology (cervical cancer screening), airport baggage screening, and breast cancer screening. False alarm (false positive) error rates typically decline at low prevalence, moving in the opposite direction from miss errors. This indicates a shift in the observer’s decision criterion. At low prevalence, observers become more reluctant to call something a target. Several studies – ours and others - have shown that this “conservative” criterion shift is not adequate to explain the entire prevalence effect. Wolfe and VanWert (2010) developed a “Dual- Threshold” model that better captures the important aspects of the prevalence effect data by proposing two effects of low prevalence: (1) the conservative shift in the criterion for deciding if an attended item is a target, and (2) a lowering of the “quitting threshold.” The quitting threshold determines when observers end a search. Quitting too soon also increases the chance that the observer will miss a target. Prevalence effects have been studied in experimental isolation from other aspects of search. However, in tasks like breast cancer screening, other factors interact with prevalence. The four projects in the present proposal each investigate one of these interactions. Project 1 examines the relationship of prevalence to the “vigilance decrements” that are seen as time elapses in a task. In search, observers must maintain an internal, mental representation of the search target (or targets). Project 2 is concerned with the impact of prevalence on these “target templates”. Advances in artificial intelligence (notably deep learning) are producing tools to assist expert searchers. However, once deployed, these AI tools have been less effective than theory predicts. Project 3 tests the hypothesis that part of the problem is another side-effect of low prevalence and the project tests a potential intervention. Finally, clinicians, searching for one type of target (e.g. pneumonia) are supposed to report signs of other possible problems (e.g. lung cancer). Project 4 probes the role of prevalence in the failure to report such “incidental findings”. Again, we test several interventions. This is “use-inspired, basic research” whose results will provide guidance for experts performing socially important low prevalence tasks.

低流行度搜索形成了一类重要且有问题的视觉搜索任务。这些都是 搜索目标很少的任务。许多具有重要社会意义的任务，例如机场安全或癌症 筛查是低流行率的任务。之前的工作（大部分来自我们实验室）已经表明，低 流行可能会产生不良影响。最值得注意的是，遗漏（假阴性）错误明显 患病率低时升高。如果搜索的目的是检测某些内容，这就是一个明显的问题 罕见但重要，就像癌症或恐怖威胁一样。我们之前的工作已经记录了这种模式 许多专家领域的遗漏错误增加，包括细胞学（宫颈癌筛查）、 机场行李检查和乳腺癌检查。误报（误报）错误率通常 发生率低时下降，与漏失错误的方向相反。这表明了转变 观察者的决策标准。在流行率较低的情况下，观察者变得更不愿意将某事物称为“ 目标。我们和其他人的多项研究表明，这种“保守”标准的转变并不 足以解释整个流行效应。 Wolfe 和 VanWert (2010) 开发了一种“双 Threshold”模型通过提出更好地捕捉流行效应数据的重要方面 低流行率的两个影响：（1）决定是否参加项目的标准的保守转变 是一个目标，以及（2）降低“戒烟门槛”。退出阈值决定何时 观察员结束搜索。太早退出也会增加观察者错过目标的机会。 流行率效应已经在与搜索的其他方面隔离的实验中进行了研究。然而， 在乳腺癌筛查等任务中，其他因素与患病率相互作用。四个项目在 目前的提案分别调查其中一种相互作用。项目 1 检查以下关系 随着任务时间的流逝，“警惕性下降”的现象普遍存在。观察者在搜寻中 必须保持搜索目标（或多个目标）的内部心理表征。关注项目2 流行率对这些“目标模板”的影响。人工智能的进步（特别是 深度学习）正在开发工具来协助专家搜索者。然而，一旦部署，这些人工智能工具 效果不如理论预测。项目 3 测试了部分问题的假设 低患病率的另一个副作用是该项目测试了潜在的干预措施。最后，临床医生， 搜索一种类型的目标（例如肺炎）应该报告其他可能的迹象 问题（例如肺癌）。项目 4 探讨了流行率在未报告此类事件中所起的作用。 “偶然发现”。我们再次测试了几种干预措施。这是“受使用启发的基础研究” 结果将为执行具有社会重要意义的低流行率任务的专家提供指导。