BIGDATA: Small: DCM: ESCE: Condensate Database for Efficient Anomaly Detection and Quality Assurance of Massive Cryospheric Data

该奖项旨在解决以前所未有的速度生成的越来越多的冰冻圈数据的问题，特别是可重复性和更高级别的数据分析，例如这些大数据流中的异常检测。这项工作建立在以前NSF授予的项目的结果之上，该项目旨在开发“数据棒技术”，其中数据被组织在时间列（又名“数据棒”）中，其中包括顺序存储的多个时间戳和传感器值，但不一定是固定的时间间隔。这将导致研究和开发用于构建“冷凝数据库”的创新技术，这些数据库比原始数据集小得多（但仍能捕获关键特征）。挑战在于，冰冻圈数据是海量的，而且多种多样，有跨越各种时空尺度的“正常”和“异常”模式。研究的三个主要领域是：（1）空间和时间上的自适应邻域阈值法;（2）压缩域模式检测和变化分析;（3）多模态、多尺度冰冻层数据的混合浓缩。开发的技术和软件将通过国家冰雪数据中心（NSIDC）现有的支持平台公开共享，并通过科学出版物和美国地球物理联盟会议等顶级会议上的演讲进行宣传。下一代科学家和技术专业人员将通过学生课程整合，远程学习和教育研讨会在大数据管理，数据分析和科学发现的跨学科领域进行培训。

This award is for addressing the problem of the increasing amount of cryospheric data being generated at an unprecedented rate, and in particular, for the discoverability and higher-level data analytics such as anomaly detection in these big data flows. This work builds on the results from the previously NSF-awarded project aimed at the development of "datarods technology", in which the data are organized in temporal columns (aka "datarods") that include multiple time stamps and sensor values stored sequentially but not necessarily as a fixed time interval. These will lead to investigation and development of innovative techniques for the construction of "Condensate Databases" that are much smaller (but yet capture key characteristics) of the original datasets. The challenges lie in the fact that cryospheric data are massive and diverse, and have "normal" and "abnormal" patterns spanning a wide range of spatial and temporal scales. Three main areas of the study are: (1) adaptive neighborhood-based thresholding in both space and time; (2) compressive-domain pattern detection and change analysis; and (3) hybrid condensation of multi-modal, multi-scale cryospheric data. The techniques and software developed will be shared publicly via the National Snow and Ice Data Center (NSIDC) existing support platform, and publicized via scientific publications and presentations at top-tier conferences such as the America Geophysical Union meetings. The next generation of scientists and technical professionals will be trained in the interdisciplinary domain of Big Data management, data analytics, and scientific discovery through the student curriculum integration, distance learning, and education workshops.