BIGDATA: Small: DCM: ESCE: Condensate Database for Efficient Anomaly Detection and Quality Assurance of Massive Cryospheric Data

点击翻译按钮获取中文摘要

This award is for addressing the problem of the increasing amount of cryospheric data being generated at an unprecedented rate, and in particular, for the discoverability and higher-level data analytics such as anomaly detection in these big data flows. This work builds on the results from the previously NSF-awarded project aimed at the development of "datarods technology", in which the data are organized in temporal columns (aka "datarods") that include multiple time stamps and sensor values stored sequentially but not necessarily as a fixed time interval. These will lead to investigation and development of innovative techniques for the construction of "Condensate Databases" that are much smaller (but yet capture key characteristics) of the original datasets. The challenges lie in the fact that cryospheric data are massive and diverse, and have "normal" and "abnormal" patterns spanning a wide range of spatial and temporal scales. Three main areas of the study are: (1) adaptive neighborhood-based thresholding in both space and time; (2) compressive-domain pattern detection and change analysis; and (3) hybrid condensation of multi-modal, multi-scale cryospheric data. The techniques and software developed will be shared publicly via the National Snow and Ice Data Center (NSIDC) existing support platform, and publicized via scientific publications and presentations at top-tier conferences such as the America Geophysical Union meetings. The next generation of scientists and technical professionals will be trained in the interdisciplinary domain of Big Data management, data analytics, and scientific discovery through the student curriculum integration, distance learning, and education workshops.