Evaluating the Impact on Clinical Task Efficiency of a Natural Language Processing Algorithm for Searching Medical Documents: Prospective Crossover Study.
Evaluating the Impact on Clinical Task Efficiency of a Natural Language Processing Algorithm for Searching Medical Documents: Prospective Crossover Study.
复制标题
DOI：
10.2196/39616
复制
发表时间：
2022-10-26
影响因子：
3.2
通讯作者：
O'Neil, Alison Q.
中科院分区：
文献类型：
--
作者：
Park, Eunsoo H.;Watson, Hannah I.;Mehendale, Felicity V.;O'Neil, Alison Q.
关键词：
从电子健康记录 (EHR) 中的自由文本中进行信息检索 (IR) 既耗时又复杂。我们假设自然语言处理 (NLP) 增强的 EHR 搜索功能可以使临床工作流程更加高效，并减少临床医生的认知负担。本研究旨在评估 3 个级别的搜索功能（无搜索、字符串搜索和 NLP 增强搜索）在模拟临床环境中从 EHR 文档的自由文本中支持临床用户 IR 的效果。通过将 3 组患者笔记上传到 EHR 研究软件应用程序并将这些笔记与 3 个相应的 IR 任务一起呈现来模拟临床环境。任务包含多项选择题和自由文本问题。采用前瞻性交叉研究设计，招募了 3 组评估者，其中包括医生（n=19）和医学生（n=16）。评估者根据随机分配的组，使用每个搜索功能按顺序执行 3 项任务。测量和分析任务完成的速度和准确性，并在反馈调查中审查用户对 NLP 增强搜索的看法。 NLP 增强搜索比字符串搜索 (5.14%; P=.02) 和无搜索 (5.13%; P=.08) 更能准确地完成任务。 NLP 增强搜索和字符串搜索促进了相似的任务速度，与无搜索功能相比，两者的速度分别提高了 11.5% (P=.008) 和 16.0% (P=.007)。总体而言，93% 的评估者认为 NLP 增强搜索将使临床工作流程比字符串搜索更高效，定性反馈报告称 NLP 增强搜索减少了认知负荷。据我们所知，这项研究是迄今为止对不同搜索功能最大规模的评估，以支持现实临床工作流程中的目标临床用户，并采用三向前瞻性交叉研究设计。与无需搜索的浏览临床记录相比，NLP 增强的搜索提高了临床 EHR IR 任务的准确性和速度。与直接搜索词匹配相比，NLP 增强的搜索提高了准确性并减少了临床 EHR IR 任务所需的搜索数量。
Information retrieval (IR) from the free text within electronic health records (EHRs) is time consuming and complex. We hypothesize that natural language processing (NLP)–enhanced search functionality for EHRs can make clinical workflows more efficient and reduce cognitive load for clinicians. This study aimed to evaluate the efficacy of 3 levels of search functionality (no search, string search, and NLP-enhanced search) in supporting IR for clinical users from the free text of EHR documents in a simulated clinical environment. A clinical environment was simulated by uploading 3 sets of patient notes into an EHR research software application and presenting these alongside 3 corresponding IR tasks. Tasks contained a mixture of multiple-choice and free-text questions. A prospective crossover study design was used, for which 3 groups of evaluators were recruited, which comprised doctors (n=19) and medical students (n=16). Evaluators performed the 3 tasks using each of the search functionalities in an order in accordance with their randomly assigned group. The speed and accuracy of task completion were measured and analyzed, and user perceptions of NLP-enhanced search were reviewed in a feedback survey. NLP-enhanced search facilitated more accurate task completion than both string search (5.14%; P=.02) and no search (5.13%; P=.08). NLP-enhanced search and string search facilitated similar task speeds, both showing an increase in speed compared to the no search function, by 11.5% (P=.008) and 16.0% (P=.007) respectively. Overall, 93% of evaluators agreed that NLP-enhanced search would make clinical workflows more efficient than string search, with qualitative feedback reporting that NLP-enhanced search reduced cognitive load. To the best of our knowledge, this study is the largest evaluation to date of different search functionalities for supporting target clinical users in realistic clinical workflows, with a 3-way prospective crossover study design. NLP-enhanced search improved both accuracy and speed of clinical EHR IR tasks compared to browsing clinical notes without search. NLP-enhanced search improved accuracy and reduced the number of searches required for clinical EHR IR tasks compared to direct search term matching.
登录
查看更多内容
DOI：
10.1136/amiajnl-2014-002726
发表时间：
2015-01
期刊：
Journal of the American Medical Informatics Association : JAMIA
影响因子：
--
作者：
Burke HB;Sessums LL;Hoang A;Becher DA;Fontelo P;Liu F;Stephens M;Pangaro LN;O'Malley PG;Baxi NS;Bunt CW;Capaldi VF 2nd;Chen JM;Cooper BA;Djuric DA;Hodge JA;Kane S;Magee C;Makary ZR;Mallory RM;Miller T;Saperstein A;Servey J;Gimbel RW
通讯作者：
Gimbel RW
DOI：
10.1016/j.ijmedinf.2017.08.005
发表时间：
2017-11-01
影响因子：
4.9
作者：
Blijleven, Vincent;Koelemeijer, Kitty;Jaspers, Monique
通讯作者：
Jaspers, Monique
影响因子：
13.8
作者：
Ruppel, Halley;Bhardwaj, Aashish;Liu, Vincent X.
通讯作者：
Liu, Vincent X.
影响因子：
4.9
作者：
Natarajan K;Stein D;Jain S;Elhadad N
通讯作者：
Elhadad N
影响因子：
2.9
作者：
Joukes, Erik;Abu-Hanna, Ameen;de Keizer, Nicolette F.
通讯作者：
de Keizer, Nicolette F.