III-COR-Small: Capturing Data Uncertainty in High-Volume Stream Processing

该项目的目标是开发一个流处理系统，该系统可以捕获从数据收集到查询处理再到最终结果生成的数据不确定性。这个项目关注的是自然建模为连续随机变量的数据。对于此类数据，它采用基于概率和统计理论的原则方法来捕获数据不确定性，并将该方法集成到大容量流处理中。该项目的第一个贡献是捕获来自传感设备的原始数据流的不确定性。由于原始数据流可能不会以适合查询处理的格式呈现数据，并且可能具有很高的噪声，因此该项目采用底层数据生成过程的概率模型和机器学习技术，将原始数据有效地转换为具有不确定性度量的所需表示。第二个贡献是捕获数据通过各种查询操作符传播时的不确定性。为了有效地量化查询运算符的结果不确定性，本项目探索了基于概率和统计理论的各种技术，以减少输入流需要携带的统计数据，并加快结果分布的计算。该项目通过课程开发将研究和教育结合起来，并通过大学外展和CRA的分布式导师计划使妇女更广泛地参与研究。该项目还包括在目标跟踪和监测以及危险天气监测等领域的软件发布和实际部署，从而产生重大的科学和社会影响。该项目的结果已在该项目的网站（http://avid.cs.umass.edu/projects/uncertain-streams/）上公布。

The goal of this project is to develop a stream processing system that captures data uncertainty from data collection to query processing to final result generation. This project focuses on data that is naturally modeled as continuous random variables. For such data, it employs a principled approach grounded in probability and statistical theory to capture data uncertainty and integrates this approach into high-volume stream processing. The first contribution of the project is to capture uncertainty of raw data streams from sensing devices. Since the raw streams may not present data in a format suitable for query processing and can be highly noisy, this project employs probabilistic models of the underlying data generation process and machine learning techniques to efficiently transform raw data into a desired representation with an uncertainty metric. The second contribution is to capture uncertainty as data propagates through various query operators. To efficiently quantify result uncertainty of a query operator, this project explores various techniques based on probability and statistical theory to reduce statistics that input streams need to carry and to expedite the computation of result distributions. This project integrates research and education through curriculum development and enables broader participation of women in research through college outreach and CRA's distributed mentor program. This project also includes software release and real-world deployments in domains such as object tracking and monitoring and hazardous weather monitoring, resulting in significant scientific and social impacts. Results of this project are disseminated at the project web site (http://avid.cs.umass.edu/projects/uncertain-streams/).