III: Medium: Collaborative Research: Supporting High-Value Analytics on Big Low-Value Data

大量的数字信息正在通过社交网络、博客、在线社区、新闻来源和移动应用程序以及无数基于设备的来源(如智能家居设备和可穿戴传感器)生成。许多领域的数据分析师，例如政府、公共卫生、国家安全和公共安全，将从对这些数据进行回顾和交互分析的能力中受益匪浅。这些数据的关键特征是，单个项目，如推文或传感器读数，本质上是低价值的。只有当大量这样的数据被放在一起分析时，这些数据才会变得高价值。该项目寻求新的数据管理技术，使数据分析师能够处理大量这样的低价值数据。关键的挑战是使用经济高效的解决方案(如廉价的商品硬件)高效和交互地支持分析查询，同时意识到数据的低价值性质。对数据分析的支持已经得到了很好的研究，无论是对于集中式数据库还是并行数据库，对于表格数据。然而，考虑到典型企业的高价值事务数据可以容纳在高端服务器的内存中的内存价格，最新的工作是对内存驻留数据进行分析。相比之下，该项目旨在支持对来自社交、移动、网络和物联网数据源的数据进行分析。这些数据要大得多，所以内存驻留对于存储或分析来说并不划算，因为只有聚集在一起，数据项才会变得高价值。该项目有三个主要推动力。第一个重点是针对嵌套的、半结构化的、缺乏预定义模式的大量数据的高效存储和资源感知查询处理。第二个推动力引入了灵活的联接框架来处理复杂的联接查询--包括空间、时间和文本数据上的联接--以允许组合多个数据集以增加它们的价值。第三个推力，由于大的低值通常涉及事件序列，专注于高效的窗口查询处理；为了进行大的低值数据分析，窗口查询的并行处理对于大的低值数据分析至关重要。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A wealth of digital information is being generated through social networks, blogs, online communities, news sources, and mobile applications as well as a myriad of device-based sources such as smart-home devices and wearable sensors. Data analysts in a number of domains, e.g., government, public health, national security, and public safety, stand to benefit greatly from the ability to perform retrospective as well as interactive analyses over such data. The key feature of this data is that an individual item, such as a tweet or a sensor reading, is low-value by nature. Such data becomes of high-value only when large quantities of such data are analyzed together. This project seeks new data management techniques to enable data analysts to process large quantities of such low-value data. The key challenge is to support analytic queries efficiently and interactively, while being aware of the low-value nature of the data, using cost-effective solutions such as cheap commodity hardware.Support for data analytics has been well studied, both for centralized and parallel databases, for tabular data. However, given memory prices where the high-value transactional data for a typical enterprise can fit in the memory of a high-end server, most recent work has been on analytics for memory-resident data. In contrast, this project aims to support analytics over data arising from social, mobile, Web, and IoT data sources. This data is much larger, so memory-residence is not cost effective for storage or analysis, as only in aggregate do the data items become high-value. The project has three main thrusts. The first thrust focuses on efficient storage and resource-aware query processing for large volumes of data that are nested, semi-structured, and lacking a predefined schema. The second thrust introduces a flexible join framework to handle complex join queries – including joins over spatial, temporal, and textual data – to allow multiple datasets to be combined to increase their value. The third thrust, since big low-value often involves sequences of events, focuses on efficient window query processing; parallel processing of window queries, in order to scale, is essential for big low-value data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.