CAREER: No Time to Explain: Developing Robots that Actively Prevent Overtrust during Emergencies

对于许多应用程序，为了安全使用机器人，用户必须知道何时以及在多大程度上信任机器人。然而，研究表明，人们往往过于信任机器人和自动化。这让他们处于危险之中，因为他们对机器人的所见、所知和所能抱有不切实际的期望。该项目的总体目标是开发能够帮助人们正确校准他们对机器人信任的机器人。该项目着眼于机器人引导的紧急疏散背景下的这个问题，目标是驻扎在建筑物内的机器人可以作为即时的第一响应者，帮助人们在紧急情况下安全撤离。该项目将为一个长期的研究项目奠定基础，该项目致力于了解导致过度信任的原因，并预测可能导致人们接受过多风险的情况。除了确保安全疏散外，这项工作还将促进自动驾驶汽车和医疗机器人的安全部署。机器人的安全使用也是一个伦理问题。因此，该项目提出了广泛的教育和推广目标，重点是使用机器人：(1)保证学生在学校的安全；(2)拓展和发展机器人伦理学相关课程；(3)教育有需要的人。该项目集中在三个目标上：1)了解导致人们过于信任机器人的情境、经验和机器人相关因素；2)感知地识别一个人是否过于信任机器人；3)生成机器人传达命令和解释，以通知和吸引撤离人员。第一个目标考察了情况的模糊性和最近与机器人的经历如何影响信任。这个目标将产生数据，这些数据将用于创建方法，使机器人能够预测一个人的信任被错误校准的可能性。第二个目标将产生一套新的感知方法，使机器人能够评估人们对机器人的信任程度，并检测人们是否过于信任机器人。最终目标将调查机器人生成的命令和解释是否以及如何用于改善机器人引导疏散期间的信任校准，开发允许机器人维护权威并提供解释的行为。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

For many applications, to safely use a robot a user must know when and how much to trust the robot. Yet, research has shown that people tend to trust robots and automation too much. This puts them at risk because they have unrealistic expectations about what a robot sees, knows, and can do. The overall goal of this project is to develop robots that can help people correctly calibrate their trust in the robot. The project looks at this problem in the context of robot-guided emergency evacuation with the goal that robots stationed inside of buildings can serve as instantaneous first responders helping people safely evacuate during an emergency. The project will lay the groundwork for a career long research program devoted to understanding the causes that underlie over-trust and predicting situations that may induce people to accept too much risk. In addition to ensuring safe evacuation, this work will also foster the safe deployment of autonomous vehicles and healthcare robots. The safe use of robots is also an ethics question. This project therefore proposes a wide range of educational and outreach objectives focused on using robots to: (1) keep students safe in schools; (2) expand and develop robot ethics-related curricula; and (3) educate those in need.This project concentrates on three objectives: 1) understanding the situational, experiential, and robot-related factors that lead people to trust a robot too much; 2) perceptually recognizing when a person is trusting a robot too much; and 3) generating robot conveyed commands and explanations to inform and engage an evacuee. The first objective examines how situation ambiguity and recent experiences with a robot influence trust. This objective will generate data that will be used to create methods allowing a robot to predict the likelihood that a person’s trust is mis-calibrated. The second objective will produce a suite of new perceptual methods allowing a robot to evaluate how much the person is trusting the robot and to detect if the person is trusting the robot too much. The final objective will investigate whether and how robot generated commands and explanations can be used to improve trust calibration during a robot-guided evacuation, developing behaviors that allow a robot to assert authority and provide explanations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.