Using AI to improve our understanding of verbal confidence and to aid decision-making: Eyewitness lineup identification as a model case

从医疗到预测累犯，人工智能（AI）在人类决策中发挥着越来越大的作用，这在很大程度上是因为AI预测在各种不同的领域都上级人类预测。 这个项目使用目击者指认作为一个范例。 该项目的一个目标是使用人工智能来更好地理解对身份的信心的口头表达。 例如，当一个目击者说“我很确定是他”关于一个列队指认，目击者的指认是正确的可能性是多少？该项目的另一个目的是研究如何最好地将人工智能输出传达给人们，以提高他们对目击者身份识别准确性的预测。 该项目包括两组实验。 一组使用目击者记忆范式的变化来检查言语的预测价值（例如，“I'm pretty certain”）和数字（例如，“I'm 75% certain”（我有75%的把握）表示信心。 在很大程度上，人们认为口头信心陈述反映了与数字信心评级相同的潜在信息。 机器学习分类器用于量化言语信心，以解释为什么言语信心与数字信心无关，但可以在预测响应的准确性方面贡献独特的附加值。 第二组实验测试两个预测。 首先，机器学习对阵容识别准确性的估计比人类对各种上下文信息影响的估计更有抵抗力。 其次，传达关于目击者识别准确性的机器学习估计的认知强迫方法在提高人们对目击者准确性的预测方面是最有效的，特别是在人们对目击者表现的直觉是错误的情况下。 总的来说，该项目解决了在人工智能辅助扩展到目击者识别领域之前识别人类算法交互中的问题的需求。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估而被认为值得支持。

From medical treatment to predicting recidivism, artificial intelligence (AI) is playing an increasingly larger role in human decision-making – in large part because AI predictions are superior to human predictions in a variety of different domains. This project uses eyewitness lineup identifications as a model paradigm. One aim of the project uses AI to better understand verbal expressions of confidence about an identification. For example, when an eyewitness states “I’m pretty sure it’s him” about a lineup identification what is the likelihood that the eyewitness’s identification is correct? Another aim of the project examines how best to convey AI output to people so as to improve their predictions about the accuracy of an eyewitness’s identification. The project consists of two sets of experiments. One set uses variations on an eyewitness memory paradigm to examine the predictive value of verbal (e.g., “I’m pretty certain”) and numeric (e.g., “I’m 75% certain”) expressions of confidence. It is largely assumed that verbal confidence statements reflect the same underlying information as numeric confidence ratings. Machine-learning classifiers are used to quantify verbal confidence to explain why verbal confidence is not redundant with numeric confidence but can contribute unique added value in predicting the accuracy of a response. A second set of experiments test two predictions. First, that machine learning estimates about the accuracy of a lineup identification are more resistant than human estimates to the effects of various kinds of contextual information. Second, that a cognitive-forcing method of conveying machine-learning estimates about eyewitness identification accuracy is most effective at improving people’s predictions about eyewitness accuracy, particularly under conditions when people’s intuitions about eyewitness performance are wrong. Overall, this project addresses a need to identify issues in human-algorithm interactions before the spread of AI-assistance to the domain of eyewitness identification.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.