Elements: Data: Integrating Human and Machine for Post-Disaster Visual Data Analytics: A Modern Media-Oriented Approach

该项目创建了一个面向科学的可视化数据服务，促进了基于可视化内容的数据集查询。该方法允许用户基于视觉相似性搜索数据，即使在故障或观察的术语还没有科学名称的情况下。 可视化分析数据和应用服务将部署在基于云的平台上。 其结果将产生一个框架，使人们能够获取和分析来自不同来源的大量图像。 研究团队创建了VISER（可视化结构专业知识复制器），它将作为一个全面的基于云的数据分析服务，并将促进用户最需要的数据和应用程序的使用和集成。 该框架将实现两个新的概念：数据即服务和应用程序即服务，这将为用户提供数据和应用程序，而无需配置软件系统或软件包。 该方法还采用人工智能来解释图像的内容。 VISER将使用卷积神经网络（CNN）来训练新类别的自定义分类器。 VISER将开发和部署三个应用程序：App1将提取相关的视觉背景，App2将促进基于相似性的视觉搜索（通过使用Siamese CNN），App3将帮助执行基于Google街景的事件前/灾前图像的自动提取。 这些工具的应用将促进自动模式识别和更有效的施工技术的科学。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

This project creates a science-oriented visual data service that facilitates the query of datasets based on visual content. The approach allows a user to search for data based on visual similarity, even in cases where a term for the failure or observation does not yet have a scientific name. The visual analysis data and application services will be deployed on a cloud-based platform. The results will produce a framework enabling access to and analysis of a large amount of imagery from diverse sources. The research team creates VISER (Visual Structural Expertise Replicator), which will serve as a comprehensive cloud-based data analytics service and will facilitate the use of and integrate data and applications most needed by the user. The framework will implement two novel concepts: data-as-a-service and applications-as-a-service, which will bring data and applications to the user without the need to configure software systems or packages. The approach also employs artificial intelligence to interpret the contents of the images. VISER will use convolutional neural networks (CNNs) to train custom classifiers for new categories. Three applications will be developed and deployed within VISER: App1 will extract relevant visual context, App2 will facilitate similarity-based visual searching (through the use of a Siamese CNN), and App3 will help perform automatic extraction of pre-event/pre-disaster images based on Google Street View. The application of these tools would advance both the science of automated pattern recognition and of more effective construction techniques.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.