Identifying Rare Circumstances Preceding Female Firearm Suicides: Validating A Large Language Model Approach.
Identifying Rare Circumstances Preceding Female Firearm Suicides: Validating A Large Language Model Approach.
复制标题
作者：
枪支自杀在男性中更为普遍，但从2010年到2020年，经年龄调整的女性枪支自杀率增加了20%，超过男性的增长率约8个百分点，女性枪支自杀可能有不同的原因。在美国，国家暴力死亡报告系统（NVDRS）是一个全面的暴力死亡数据来源，包括来自验尸官或法医和执法部门的非结构化事件叙述报告。传统的自然语言处理方法已被用于识别女性枪支自杀死亡之前的常见情况，但由于训练数据不足，未能识别出罕见的情况。本研究旨在利用大型语言模型方法，在NVDRS中提供的非结构化验尸官或法医和执法叙述报告中识别女性枪支自杀前的罕见情况。我们使用了2014年至2018年NVDRS中1462名女性枪支自杀死者的叙述性报告。报告是用英文写的。我们对女性枪支自杀前的9种罕见情况进行了编码。我们通过利用大型语言模型方法以是/否问答格式来预测这些情况。我们用F1-score（0 - 1）来衡量预测准确性。F1分数是精确度（阳性预测值）和召回率（真阳性率或灵敏度）的调和平均值。我们的大型语言模型远远优于传统的支持向量机监督机器学习方法。与支持向量机模型相比，对于大多数不常见的情况，其F1分数小于0.2，我们的大型语言模型方法对于4种情况和2种情况的F1分数分别超过0.6和0.8。大型语言模型方法的使用显示出了希望。有兴趣使用自然语言处理来识别叙述性报告数据中不常见情况的研究人员可能会受益于大型语言模型。
Firearm suicide has been more prevalent among males, but age-adjusted female firearm suicide rates increased by 20% from 2010 to 2020, outpacing the rate increase among males by about 8 percentage points, and female firearm suicide may have different contributing circumstances. In the United States, the National Violent Death Reporting System (NVDRS) is a comprehensive source of data on violent deaths and includes unstructured incident narrative reports from coroners or medical examiners and law enforcement. Conventional natural language processing approaches have been used to identify common circumstances preceding female firearm suicide deaths but failed to identify rarer circumstances due to insufficient training data. This study aimed to leverage a large language model approach to identify infrequent circumstances preceding female firearm suicide in the unstructured coroners or medical examiners and law enforcement narrative reports available in the NVDRS. We used the narrative reports of 1462 female firearm suicide decedents in the NVDRS from 2014 to 2018. The reports were written in English. We coded 9 infrequent circumstances preceding female firearm suicides. We experimented with predicting those circumstances by leveraging a large language model approach in a yes/no question-answer format. We measured the prediction accuracy with F1-score (ranging from 0 to 1). F1-score is the harmonic mean of precision (positive predictive value) and recall (true positive rate or sensitivity). Our large language model outperformed a conventional support vector machine–supervised machine learning approach by a wide margin. Compared to the support vector machine model, which had F1-scores less than 0.2 for most infrequent circumstances, our large language model approach achieved an F1-score of over 0.6 for 4 circumstances and 0.8 for 2 circumstances. The use of a large language model approach shows promise. Researchers interested in using natural language processing to identify infrequent circumstances in narrative report data may benefit from large language models.