Developing an artificial intelligence-based headache diagnostic model and its utility for non-specialists' diagnostic accuracy
Developing an artificial intelligence-based headache diagnostic model and its utility for non-specialists' diagnostic accuracy
复制标题
DOI：
10.1177/03331024231156925
复制
发表时间：
2023-05-01
期刊：
影响因子：
4.9
通讯作者：
Takeshima, Takao
中科院分区：
文献类型：
--
作者：
Katsuki, Masahito;Shimazu, Tomokazu;Takeshima, Takao
背景头痛疾病的误诊是一个严重的问题。因此，我们在一家专门的头痛医院使用大型问卷数据库开发了一个基于人工智能的头痛诊断模型。方法第1阶段：我们基于对头痛专家诊断的4000名患者（2800个训练数据集和1200个测试数据集）的回顾性调查开发了一个人工智能模型。第二阶段：验证模型的有效性和准确性。五名非头痛专家首先诊断了50名患者的头痛，然后使用AI重新诊断。事实是头痛专家的诊断。评估了头痛专家和有或没有人工智能的非专家之间的诊断性能和一致率。结果第1阶段：对于测试数据集，该模型的宏观平均准确度、灵敏度（召回率）、特异度、精确度和F值分别为76.25%、56.26%、92.16%、61.24%和56.88%。第二阶段：五名非专家在没有人工智能的情况下诊断出头痛，总体准确率为46%，地面真相为0.212 kappa。人工智能的统计学改善值分别为83.20%和0.678。其他诊断指标也有所改善。结论人工智能提高了非专家诊断性能。鉴于该模型基于单一中心数据的局限性以及继发性头痛的诊断准确性较低，需要进一步收集和验证数据。
BackgroundMisdiagnoses of headache disorders are a serious issue. Therefore, we developed an artificial intelligence-based headache diagnosis model using a large questionnaire database in a specialized headache hospital. MethodsPhase 1: We developed an artificial intelligence model based on a retrospective investigation of 4000 patients (2800 training and 1200 test dataset) diagnosed by headache specialists. Phase 2: The model's efficacy and accuracy were validated. Five non-headache specialists first diagnosed headaches in 50 patients, who were then re-diagnosed using AI. The ground truth was the diagnosis by headache specialists. The diagnostic performance and concordance rates between headache specialists and non-specialists with or without artificial intelligence were evaluated. ResultsPhase 1: The model's macro-average accuracy, sensitivity (recall), specificity, precision, and F values were 76.25%, 56.26%, 92.16%, 61.24%, and 56.88%, respectively, for the test dataset. Phase 2: Five non-specialists diagnosed headaches without artificial intelligence with 46% overall accuracy and 0.212 kappa for the ground truth. The statistically improved values with artificial intelligence were 83.20% and 0.678, respectively. Other diagnostic indexes were also improved. ConclusionsArtificial intelligence improved the non-specialist diagnostic performance. Given the model's limitations based on the data from a single center and the low diagnostic accuracy for secondary headaches, further data collection and validation are needed.