III: Small: From Real-Time Sensor Data Streams to Continuous Data Fields Models: Formal Foundations and Computational Challenges

III：小：从实时传感器数据流到连续数据字段模型：形式基础和计算挑战

• 批准号: 1527504
• 负责人: Silvia Nittel
• 金额: $ 50万
• 依托单位: University of Maine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1527504&HistoricalAwards=false
• 关键词: III; Small; Real; Time; Sensor

Massive sensor data streams are created from the automatic collection of sensor data in high frequency and in near real-time today. This project aims to advance the analytical potential of live-streamed data, historical data streams, and model simulations by creating an overarching representation in the form of the field data model with a set of operators that establish the field algebra. A field is best explained as, for example, a magnetic field; the magnetic force can be determined for each point in a magnetic field and the field is therefore considered to be continuous. Similarly, environmental phenomena such as air pollution or flooding are considered continuous in space and time although they are sampled at limited, discrete time-space locations with sensors. This project develops the field algebra, which is an intuitive, yet mathematically defined formalism to represent real-world phenomena as fields and to express analytical needs as canonical operations over fields. The field model represents phenomena as continuous entities again, and the implementation hides the fact that their spatio-temporal continuity is calculated on-the-fly based on real-time measurements streams. Extending sensor data streams to fields is transformative, as rarely a domain scientist is interested in the readings of individual sensors. Allowing scientists to work with high-level abstractions will significantly enhance their analytical tasks such as finding insights about changes, trends, or unexpected events happening in the real world. The project will integrate fields and data streams mathematically so that mappings between both are well defined. The field data model is complemented by the development of an innovative computational framework for synthesizing and analyzing fields based on very large numbers of high throughput, real-time sensor data streams, and for creating continuous representations on-the-fly. This framework provides novel algorithms to assure that the field operators can absorb the throughput of very large numbers of sensor data streams, yet still compute complex analytical results in near real-time. This project will benefit our society by enabling us to react to situations such as extreme weather events, environmental disasters or chemical accidents immediately, and organize response effort based on accurate and timely information; this will help to protect the public interests better. The research in this project develops a formal foundation for sensor data streams by abstracting them as geographic fields, and a scalable computational framework that computes field operators on massive numbers of sensor data streams in near real-time. In this research, the field algebra, with a recursive definition of fields and a set of field operators are formalized. The field algebra and data streams are formally integrated on the level of their mathematical foundations. The formal field algebra is implemented as a data type hierarchy and integrated with stream data models. At the same time, a computational framework is developed that extends data stream engines with computational components to estimate spatio-temporal fields based on recursive or transposed field definitions, and the evaluation of complex predicates over fields, which lays the foundation for co-analyzing live and historic fields. The results of this project will be distributed via scientific publications, open source software, and online training tutorials and classes. The project web site (https://silvianittel.wordpress.com/from-streams-to-fields-nsf/) will provide access to the results of this project.

如今，大量的传感器数据流是从高频率和近实时的传感器数据自动收集中创建的。该项目旨在通过使用一组建立场代数的运算符以场数据模型的形式创建总体表示来提高实时流数据，历史数据流和模型模拟的分析潜力。场最好解释为，例如，磁场;可以确定磁场中每个点的磁力，因此场被认为是连续的。类似地，空气污染或洪水等环境现象被认为在空间和时间上是连续的，尽管它们是在有限的离散时空位置用传感器采样的。该项目开发了场代数，这是一种直观的，但数学定义的形式主义，以表示作为场的现实世界的现象，并表示作为规范操作的分析需求领域。场模型再次将现象表示为连续实体，并且实现隐藏了基于实时测量流在运行中计算其时空连续性的事实。将传感器数据流扩展到领域是变革性的，因为领域科学家很少对单个传感器的读数感兴趣。允许科学家使用高级抽象将显著增强他们的分析任务，例如发现有关真实的世界中发生的变化，趋势或意外事件的见解。该项目将以数学方式集成字段和数据流，以便两者之间的映射得到很好的定义。现场数据模型的补充是一个创新的计算框架的发展，合成和分析领域的基础上非常大量的高吞吐量，实时传感器数据流，并创建连续表示的飞行。该框架提供了新的算法，以确保现场操作员可以吸收非常大量的传感器数据流的吞吐量，但仍然在近实时计算复杂的分析结果。该项目将使我们能够立即对极端天气事件、环境灾害或化学事故等情况做出反应，并根据准确和及时的信息组织应对工作，从而造福于我们的社会，这将有助于更好地保护公共利益。该项目的研究通过将传感器数据流抽象为地理领域，为传感器数据流开发了一个正式的基础，并开发了一个可扩展的计算框架，可以近实时地计算大量传感器数据流的领域操作员。本文给出了域代数的形式化描述，给出了域的递归定义和一组域算子。域代数和数据流的数学基础的水平上正式集成。形式域代数实现为数据类型层次结构，并与流数据模型集成。同时，开发了一个计算框架，该框架扩展了数据流引擎的计算组件，以基于递归或转置的字段定义来估计时空字段，并对字段上的复杂谓词进行评估，这为实时和历史字段的协同分析奠定了基础。该项目的成果将通过科学出版物、开源软件和在线培训教程和课程进行传播。项目网站（https：//silvianittel.wordpress.com/from-streams-to-fields-nsf/）将提供该项目的成果。