Using AI to improve our understanding of verbal confidence and to aid decision-making: Eyewitness lineup identification as a model case

从医疗到预测再犯，人工智能（AI）在人类决策中发挥着越来越大的作用，这在很大程度上是因为人工智能的预测在许多不同领域都优于人类的预测。这个项目使用目击者指认作为一个模型范例。该项目的一个目标是使用人工智能来更好地理解对身份的信心的口头表达。例如，当一个目击证人在指认指认时说“我很确定是他”目击证人指认正确的可能性有多大？该项目的另一个目的是研究如何最好地将人工智能输出传达给人们，以提高他们对目击者身份识别准确性的预测。这个项目包括两组实验。一组使用目击者记忆范式的变体来检验口头（例如，“我很确定”）和数字（例如，“我有75%的把握”）表达信心的预测价值。很大程度上认为，口头自信陈述反映了与数字自信评级相同的潜在信息。机器学习分类器用于量化口头自信，以解释为什么口头自信与数字自信并不冗余，但可以在预测响应的准确性方面贡献独特的附加价值。第二组实验验证了两个预测。首先，机器学习对阵容识别准确性的估计比人类对各种上下文信息影响的估计更有抵抗力。第二，用认知强迫的方法来传达机器学习对目击者识别准确性的估计，在提高人们对目击者准确性的预测方面最为有效，尤其是在人们对目击者表现的直觉是错误的情况下。总体而言，该项目解决了在人工智能辅助传播到目击者识别领域之前识别人类算法交互中的问题的需要。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

From medical treatment to predicting recidivism, artificial intelligence (AI) is playing an increasingly larger role in human decision-making – in large part because AI predictions are superior to human predictions in a variety of different domains. This project uses eyewitness lineup identifications as a model paradigm. One aim of the project uses AI to better understand verbal expressions of confidence about an identification. For example, when an eyewitness states “I’m pretty sure it’s him” about a lineup identification what is the likelihood that the eyewitness’s identification is correct? Another aim of the project examines how best to convey AI output to people so as to improve their predictions about the accuracy of an eyewitness’s identification. The project consists of two sets of experiments. One set uses variations on an eyewitness memory paradigm to examine the predictive value of verbal (e.g., “I’m pretty certain”) and numeric (e.g., “I’m 75% certain”) expressions of confidence. It is largely assumed that verbal confidence statements reflect the same underlying information as numeric confidence ratings. Machine-learning classifiers are used to quantify verbal confidence to explain why verbal confidence is not redundant with numeric confidence but can contribute unique added value in predicting the accuracy of a response. A second set of experiments test two predictions. First, that machine learning estimates about the accuracy of a lineup identification are more resistant than human estimates to the effects of various kinds of contextual information. Second, that a cognitive-forcing method of conveying machine-learning estimates about eyewitness identification accuracy is most effective at improving people’s predictions about eyewitness accuracy, particularly under conditions when people’s intuitions about eyewitness performance are wrong. Overall, this project addresses a need to identify issues in human-algorithm interactions before the spread of AI-assistance to the domain of eyewitness identification.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.