III: Medium: Collaborative Research: Supporting High-Value Analytics on Big Low-Value Data

通过社交网络、博客、在线社区、新闻源和移动的应用以及无数基于设备的源（例如智能家居设备和可穿戴传感器）生成大量数字信息。在许多领域的数据分析师，例如，政府、公共卫生、国家安全和公共安全将极大地受益于对这些数据执行回顾性和交互式分析的能力。这些数据的关键特征是，单个项目，如推文或传感器阅读，本质上是低价值的。只有当大量的此类数据被一起分析时，此类数据才具有高价值。该项目寻求新的数据管理技术，使数据分析师能够处理大量此类低价值数据。关键的挑战是如何有效地和交互地支持分析查询，同时意识到数据的低价值性质，使用具有成本效益的解决方案，如廉价的商品硬件。对数据分析的支持已经得到了很好的研究，包括集中式和并行数据库，以及表格数据。然而，考虑到内存价格，典型企业的高价值事务数据可以容纳在高端服务器的内存中，最近的工作一直是对内存驻留数据的分析。相比之下，该项目旨在支持对来自社交、移动的、Web和物联网数据源的数据进行分析。这些数据要大得多，因此内存驻留对于存储或分析来说并不划算，因为只有在聚合时数据项才变得有价值。该项目有三个主要目标。第一个重点是针对嵌套、半结构化和缺乏预定义模式的大量数据的高效存储和资源感知查询处理。第二个重点是引入了一个灵活的连接框架来处理复杂的连接查询-包括空间，时间和文本数据的连接-以允许组合多个数据集以增加其价值。第三个重点是，由于大低值通常涉及事件序列，因此重点关注高效的窗口查询处理;为了扩展，窗口查询的并行处理对于大低值数据分析至关重要。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

A wealth of digital information is being generated through social networks, blogs, online communities, news sources, and mobile applications as well as a myriad of device-based sources such as smart-home devices and wearable sensors. Data analysts in a number of domains, e.g., government, public health, national security, and public safety, stand to benefit greatly from the ability to perform retrospective as well as interactive analyses over such data. The key feature of this data is that an individual item, such as a tweet or a sensor reading, is low-value by nature. Such data becomes of high-value only when large quantities of such data are analyzed together. This project seeks new data management techniques to enable data analysts to process large quantities of such low-value data. The key challenge is to support analytic queries efficiently and interactively, while being aware of the low-value nature of the data, using cost-effective solutions such as cheap commodity hardware.Support for data analytics has been well studied, both for centralized and parallel databases, for tabular data. However, given memory prices where the high-value transactional data for a typical enterprise can fit in the memory of a high-end server, most recent work has been on analytics for memory-resident data. In contrast, this project aims to support analytics over data arising from social, mobile, Web, and IoT data sources. This data is much larger, so memory-residence is not cost effective for storage or analysis, as only in aggregate do the data items become high-value. The project has three main thrusts. The first thrust focuses on efficient storage and resource-aware query processing for large volumes of data that are nested, semi-structured, and lacking a predefined schema. The second thrust introduces a flexible join framework to handle complex join queries – including joins over spatial, temporal, and textual data – to allow multiple datasets to be combined to increase their value. The third thrust, since big low-value often involves sequences of events, focuses on efficient window query processing; parallel processing of window queries, in order to scale, is essential for big low-value data analytics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.