CollabLearn: An Uncertainty-Aware Crowd-AI Collaboration System for Cultural Heritage Damage Assessment
CollabLearn: An Uncertainty-Aware Crowd-AI Collaboration System for Cultural Heritage Damage Assessment
复制标题
协同学习（CollabLearn）：一种用于文化遗产损害评估的不确定性感知人机协同系统
DOI：
10.1109/tcss.2021.3109143
复制
发表时间：
2022-10
影响因子：
5
通讯作者：
Yang Zhang;Ruohan Zong;Ziyi Kou;Lanyu Shang;Dong Wang
中科院分区：
文献类型：
--
作者：
Yang Zhang;Ruohan Zong;Ziyi Kou;Lanyu Shang;Dong Wang
文化遗产地是珍贵而脆弱的资源，在我们的社会中具有重要的历史，美学和社会价值。然而，越来越频繁和严重的自然和人为灾害不断给文化遗址造成重大破坏。在这篇文章中，我们关注文化遗产损害评估（CHDA）问题，其目标是通过探索人工智能和来自众包系统的人类智能的集体优势，使用灾难事件期间社交媒体上发布的图像数据准确定位文化遗产遗址的受损区域。与其他基于基础设施的解决方案不同，社交媒体平台提供了更普遍和可扩展的解决方案，可以在灾难事件期间及时获取文化遗产损坏信息。我们的工作是受当前人工智能解决方案的局限性所驱动的，由于缺乏必要的人类文化知识来区分各种损坏类型并确定损坏的实际原因，这些解决方案无法准确地模拟复杂的文化遗产损坏。在解决我们的问题时存在两个关键的技术挑战：1）在没有地面真值标签的情况下，有效地检测人工智能的有问题的文化遗产损害估计是具有挑战性的; 2）从潜在的不可靠的人群工作者那里获得准确的文化背景知识以有效地解决人工智能的失败案例是非常重要的。为了解决上述挑战，我们开发了CollabLearn，这是一个具有不确定性的众包AI协作评估系统，它明确地探索了来自众包系统的人类智能，以识别和修复AI故障案例，并提高CHDA应用中的损坏评估准确性。对真实世界数据集的评估结果表明，CollabLearn在准确评估最近灾害事件中几个世界知名文化遗产的破坏方面始终优于最先进的人工智能和人群人工智能混合基线。
Cultural heritage sites are precious and fragile resources that hold significant historical, esthetic, and social values in our society. However, the increasing frequency and severity of natural and man-made disasters constantly strike the cultural heritage sites with significant damages. In this article, we focus on a cultural heritage damage assessment (CHDA) problem where the goal is to accurately locate the damaged area of a cultural heritage site using the imagery data posted on social media during a disaster event by exploring the collective strengths of both AI and human intelligence from crowdsourcing systems. Unlike other infrastructure-based solutions, social media platforms provide a more pervasive and scalable solution to acquire timely cultural heritage damage information during disaster events. Our work is motivated by the limitation of current AI solutions that fail to accurately model the complex cultural heritage damage due to the lack of essential human cultural knowledge to differentiate various damage types and identify the actual causes of the damage. Two critical technical challenges exist in solving our problem: 1) it is challenging to effectively detect the problematic cultural heritage damage estimation of AI in the absence of ground truth labels and 2) it is nontrivial to acquire accurate cultural background knowledge from the potentially unreliable crowd workers to effectively address the failure cases of AI. To address the above-mentioned challenges, we develop CollabLearn, an uncertainty-aware crowd-AI collaborative assessment system that explicitly explores the human intelligence from crowdsourcing systems to identify and fix AI failure cases and boost the damage assessment accuracy in CHDA applications. The evaluation results on real-world datasets show that CollabLearn consistently outperforms both the state-of-the-art AI-only and crowd-AI hybrid baselines in accurately assessing the damage of several world-renowned cultural heritage sites in recent disaster events.