CHS: Small: DeepCrowd: A Crowd-assisted Deep Learning-based Disaster Scene Assessment System with Active Human-AI Interactions

人工智能(AI)的最新进展已经改变了现代生活的许多重要领域，如交通、金融、教育、医疗保健和娱乐。本项目研究了人工智能在灾害现场评估中的应用。对于DSA，人工智能可以用于在地震、飓风或山体滑坡等灾难发生后，从图像报告中自动识别受影响地区的破坏严重程度。基于人工智能的技术的一个关键限制是许多当代模型的黑箱性质，以及随之而来的对结果和失败缺乏可解释性。该项目通过主动的人群-AI交互将人类智能与机器智能相结合，研究故障排除、调整并最终改进黑盒AI算法的问题。这项工作是对目前流行的人工智能解决方案的补充，这些解决方案主要专注于人工智能模型设计和训练样本收集。这一项目的成果将开启前所未有的机遇，在各个众助AI应用领域充分挖掘人群智慧。该项目还将为STEM的学生和来自代表性不足的群体的学生提供机会，学习人工智能与人类之间的互动。该项目开发了一个DeepCrowd框架，可用于指导未来的群组AI应用程序的设计、开发和实施，其中从人群中获得的人类智能与AI深度学习模型紧密集成，以显著提高系统性能，而不是仅限AI或仅限人类的解决方案。该项目使用了一种跨学科的方法，灵感来自人工智能、机器学习、估计理论和网络-人类互动的技术，解决了人工智能和DeepCrowd中的众包平台的黑盒挑战。具体地说，研究包括：i)开发一个群体任务生成方案，以有效地查询众包平台以获取反馈；ii)创建一个新的自适应机制，以激励人群做出及时和准确的反应；iii)设计一个交互注意力神经网络方案，使人群和人工智能模型之间能够直接交互；以及iv)开发一个群体和人工智能集成引擎，该引擎有效地结合来自人群的反馈，以缓解人工智能的失败场景。由此产生的DeepCrowd框架具有变革性，因为它将产生一套新的人群-AI交互模型和技术来构建新的人群辅助AI应用程序，并提高系统性能。该项目开发了一个DeepCrowd框架，可用于指导未来群组AI应用程序的设计、开发和实施，其中从人群中获得的人类智能与AI深度学习模型紧密集成，从而显著提高系统性能，而不是仅限AI或仅限人类的解决方案。该项目使用了一种跨学科的方法，灵感来自人工智能、机器学习、估计理论和网络-人类互动的技术，解决了人工智能和DeepCrowd中的众包平台的黑盒挑战。具体地说，研究包括：i)开发一个群体任务生成方案，以有效地查询众包平台以获取反馈；ii)创建一个新的自适应机制，以激励人群做出及时和准确的反应；iii)设计一个交互注意力神经网络方案，使人群和人工智能模型之间能够直接交互；以及iv)开发一个群体和人工智能集成引擎，该引擎有效地结合来自人群的反馈，以缓解人工智能的失败场景。由此产生的DeepCrowd框架具有变革性，因为它将产生一套新的群-AI交互模型和技术，以构建新的群辅助AI应用程序，并提升系统性能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in artificial intelligence (AI) have transformed many important domains of modern life such as transportation, finance, education, healthcare, and entertainment. This project addresses application of AI to disaster scene assessment (DSA). For DSA, artificial intelligence can be used to automatically identify damage severity of impacted areas from imagery reports in the aftermath of a disaster such as earthquake, hurricane, or landslides. A key limitation of AI based techniques is the black-box nature of many contemporary models and the consequent lack of interpretability of the results and failures. This project investigates the problem of troubleshooting, tuning, and eventually improving the black-box AI algorithms by integrating human intelligence with machine intelligence through active crowd-AI interactions. The work complements the prevailing AI solutions that primarily focus on AI model design and training sample collection. The results from this project will open up unprecedented opportunities of fully exploring the wisdom from the crowd in various crowd-assisted AI application domains. This project will also provide opportunities for students in STEM and from underrepresented groups to study the interaction between AI and humans. This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This project develops a DeepCrowd framework that can be used to guide the design, development, and implementation of future crowd-AI applications where the human intelligence obtained from the crowd is tightly integrated with AI deep learning models to significantly improve the system performance over the AI-only or human-only solutions. The project addresses the black-box challenges of AI and the crowdsourcing platform in DeepCrowd using an interdisciplinary approach inspired by techniques from AI, machine learning, estimation theory, and cyber-human interactions. In particular, the research includes i) developing a crowd task generation scheme to effectively query the crowdsourcing platform for feedback; ii) creating a novel adaptive mechanism to incentivize the crowd for timely and accurate response; iii) designing an interactive attention neural network scheme that enables direct interaction between crowd and AI models; and iv) developing a crowd and AI integration engine that effectively incorporates feedback from crowd to alleviate failure scenarios of AI. The resulting DeepCrowd framework is transformative in that it will produce a set of new crowd-AI interaction models and techniques to build novel crowd-assisted AI applications with boosted system performance.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.