CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions

人工智能（AI）的最新进展已经改变了现代生活的许多重要领域，如交通、金融、教育、医疗保健和娱乐。本项目涉及人工智能在灾难现场评估（DSA）中的应用。对于DSA，人工智能可用于在地震、飓风或山体滑坡等灾难发生后，从图像报告中自动识别受影响地区的破坏严重程度。基于人工智能的技术的一个关键限制是许多当代模型的黑箱性质，以及由此导致的结果和失败缺乏可解释性。该项目通过积极的人群-人工智能交互，将人类智能与机器智能相结合，研究故障排除、调优并最终改进黑箱人工智能算法的问题。这项工作补充了主要关注人工智能模型设计和训练样本收集的主流人工智能解决方案。该项目的成果将为在各个人群辅助人工智能应用领域充分挖掘人群智慧提供前所未有的机会。该项目还将为STEM专业的学生和代表性不足的群体提供机会，研究人工智能与人类之间的互动。该项目开发了一个DeepCrowd框架，可用于指导未来人群-人工智能应用的设计、开发和实现，其中从人群中获得的人类智能与人工智能深度学习模型紧密集成，以显著提高系统性能，优于人工智能或人工智能解决方案。该项目采用跨学科方法解决人工智能和DeepCrowd众包平台的黑箱挑战，该方法受到人工智能、机器学习、估计理论和网络-人类交互技术的启发。具体而言，研究包括：1)开发一种众包任务生成方案，以有效地查询众包平台以获得反馈；Ii)创造一种新的适应机制，激励人群做出及时准确的反应；iii)设计交互式注意力神经网络方案，实现人群与人工智能模型之间的直接交互；iv)开发人群与人工智能集成引擎，有效整合人群反馈，缓解人工智能故障场景。由此产生的DeepCrowd框架具有变革性，因为它将产生一套新的人群-人工智能交互模型和技术，以构建具有提升系统性能的新型人群辅助人工智能应用程序。该项目开发了一个DeepCrowd框架，可用于指导未来人群-人工智能应用的设计、开发和实现，其中从人群中获得的人类智能与人工智能深度学习模型紧密集成，以显著提高系统性能，优于人工智能或人工智能解决方案。该项目采用跨学科方法解决人工智能和DeepCrowd众包平台的黑箱挑战，该方法受到人工智能、机器学习、估计理论和网络-人类交互技术的启发。具体而言，研究包括：1)开发一种众包任务生成方案，以有效地查询众包平台以获得反馈；Ii)创造一种新的适应机制，激励人群做出及时准确的反应；iii)设计交互式注意力神经网络方案，实现人群与人工智能模型之间的直接交互；iv)开发人群与人工智能集成引擎，有效整合人群反馈，缓解人工智能故障场景。由此产生的DeepCrowd框架具有变革性，因为它将产生一套新的人群-人工智能交互模型和技术，以构建具有提升系统性能的新型人群辅助人工智能应用程序。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in artificial intelligence (AI) have transformed many important domains of modern life such as transportation, finance, education, healthcare, and entertainment. This project addresses application of AI to disaster scene assessment (DSA). For DSA, artificial intelligence can be used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster such as earthquake, hurricane, or landslides. A key limitation of AI based techniques is the black-box nature of many contemporary models and the consequent lack of interpretability of the results and failures. This project investigates the problem of troubleshooting, tuning, and eventually improving the black-box AI algorithms by integrating human intelligence with machine intelligence through active crowd-AI interactions. The work complements the prevailing AI solutions that primarily focus on AI model design and training sample collection. The results from this project will open up unprecedented opportunities of fully exploring the wisdom from the crowd in various crowd-assisted AI application domains. This project will also provide opportunities for students in STEM and from underrepresented groups to study the interaction between AI and humans. This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.