CAREER: Towards Exploratory Data Science on Spatio-temporal Big Data

点击翻译按钮获取中文摘要

The OPEN government data act helped in making hundreds of thousands of datasets publicly available to the scientific community and the general public; geospatial data comprise over 60% of this data. This project describes basic research towards building an end-to-end system that allows data science students and domain scientists to interactively explore spatio-temporal datasets. The overarching goal is to bridge the gap between domain scientists and data providers. On one end, it helps domain scientists in various fields, e.g., agriculture, environmental science, and political science, who have little programming skills to explore and access publicly available geospatial and temporal data. On the other end, it helps data providers, e.g., government agencies, non-profit organizations, and national research labs, to attract more data scientists to exploit and utilize public data. The project will also establish educational activities that encourage domain scientists to use public open geospatial data which will promote the reproducibility of scientific results.This project introduces new research directions that are geared towards building an end-to-end interactive exploratory system that will allow domain scientists to process, analyze, and visualize petabytes of spatio-temporal data. It consists of three research components. First, an interactive query processor provides a real-time answer to exploratory queries so that the user will stay engaged and active which increases the productivity; this innovation systematically studies approximate query processing for large geospatial data and will utilize deep learning to provide accurate error bounds for both vector and raster data for complex spatio-temporal queries. Second, to provide users with an exploratory interface, a spatio-temporal visualization component provides an interactive map-based interface that allows users to explore the spatial and temporal attributes of the data. This visualization component will also provide guided assistance to users when exploring big datasets through a novel recommendation system. Lastly, as the datasets grow in size, a dynamic storage system will continuously consume the new records and update the spatio-temporal indexes, data summaries, and visualizations on top of a distributed storage engine which is inherently immutable, i.e., does not support updates. Additionally, this project will build a working prototype where scientists can interactively explore and share spatio-temporal data and share public open data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.