Elements: Data: Integrating Human and Machine for Post-Disaster Visual Data Analytics: A Modern Media-Oriented Approach

该项目创建了一个面向科学的可视化数据服务，方便了基于可视化内容的数据集查询。该方法允许用户基于视觉相似性来搜索数据，即使在失败或观察的术语还没有科学名称的情况下也是如此。可视化分析数据和应用服务将部署在基于云的平台上。结果将产生一个框架，使人们能够获取和分析来自不同来源的大量图像。研究团队创建了VISER(视觉结构专业知识复制器)，它将作为基于云的全面数据分析服务，并将促进用户最需要的数据和应用程序的使用和集成。该框架将实现两个新的概念：数据即服务和应用即服务，这将为用户带来数据和应用，而不需要配置软件系统或包。该方法还使用人工智能来解释图像的内容。Viser将使用卷积神经网络(CNN)来训练新类别的自定义分类器。将在VISER内开发和部署三个应用程序：App1将提取相关的视觉背景，App2将促进基于相似性的视觉搜索(通过使用暹罗CNN)，App3将帮助执行基于谷歌街景的事前/灾前图像自动提取。这些工具的应用将促进自动模式识别的科学和更有效的建筑技术。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project creates a science-oriented visual data service that facilitates the query of datasets based on visual content. The approach allows a user to search for data based on visual similarity, even in cases where a term for the failure or observation does not yet have a scientific name. The visual analysis data and application services will be deployed on a cloud-based platform. The results will produce a framework enabling access to and analysis of a large amount of imagery from diverse sources. The research team creates VISER (Visual Structural Expertise Replicator), which will serve as a comprehensive cloud-based data analytics service and will facilitate the use of and integrate data and applications most needed by the user. The framework will implement two novel concepts: data-as-a-service and applications-as-a-service, which will bring data and applications to the user without the need to configure software systems or packages. The approach also employs artificial intelligence to interpret the contents of the images. VISER will use convolutional neural networks (CNNs) to train custom classifiers for new categories. Three applications will be developed and deployed within VISER: App1 will extract relevant visual context, App2 will facilitate similarity-based visual searching (through the use of a Siamese CNN), and App3 will help perform automatic extraction of pre-event/pre-disaster images based on Google Street View. The application of these tools would advance both the science of automated pattern recognition and of more effective construction techniques.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.