CAREER: A Framework for Ad Hoc Model Construction in Data Streaming Environments

职业：数据流环境中的临时模型构建框架

• 批准号: 1553685
• 负责人: Sangmi Pallickara
• 金额: $ 49.12万
• 依托单位: Colorado State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-15 至 2022-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1553685&HistoricalAwards=false
• 关键词: CAREER; Framework; Ad; Hoc; Model

Over the past decade there has been an exponential growth in data volumes driven in part by data streams generated by computer programs and observational equipment such as satellites, radars, and ecological sensors. Given the data volumes, it can be difficult to harness the data to understand phenomena and/or to make forecasts. Fitting models to the observational data is one way to accomplish this. A precursor to building such models is extracting features from the data. Models constructed using such features can then be used to predict what the outcome will be and when it is likely to happen. This research will provide scientists and researchers the tools needed to make sense of data streams generated in streaming environments. Domains where this research is broadly applicable include smart cities, traffic planning, homeland security, and ecological monitoring. The project includes an educational component focused on increasing female student participation in college STEM majors. Therefore, this project aligns with the NSF mission to promote the progress of science, to advance the national health, prosperity and welfare, and to secure the national defense.The project aims to carry out research to create an enabling infrastructure to support the generation, assessment, and refinement of ad hoc models from voluminous, multidimensional, time-series observational data at scale. Challenges in ad hoc model creation stem from the combinatorially explosive number of ways in which models can be realized. The framework, Synapse, aims to support and simplify the naturally iterative and interactive model building process over voluminous streaming data. Modelers will only need to specify a basic set of bootstrap parameters; the framework will manage complexities relating to: (1) how streams are dispersed, (2) how data accesses are managed, (3) coping with I/O and memory contentions, and (4) dispersion of model generation workloads. The research involves scalable techniques for data dispersion employing distributed hash table data structures, map-reduce-based workflows and orchestration of model creation workloads, training data management, and interactive visual assessment of model performance. A visualization component will allow modelers to quickly and effectively assess the quality of a multiplicity of models each possibly covering a different portion of the input feature space and to use these assessments to guide decisions about selection, updates or replacements of models. If successful, the framework will scale with increases in data volumes, the number of available data streams, model generation workloads, and live model evaluations.

在过去的十年中，数据量呈指数级增长，部分原因是计算机程序和观测设备（如卫星，雷达和生态传感器）产生的数据流。鉴于数据量，可能难以利用数据来理解现象和/或进行预测。根据观测数据拟合模型是实现这一目标的一种方法。建立这种模型的前提是从数据中提取特征。然后，使用这些特征构建的模型可以用来预测结果会是什么以及何时可能发生。 这项研究将为科学家和研究人员提供理解流媒体环境中生成的数据流所需的工具。这项研究广泛适用的领域包括智慧城市、交通规划、国土安全和生态监测。该项目包括一个教育组成部分，重点是增加女学生在大学STEM专业的参与。因此，本项目符合NSF促进科学进步、促进国民健康、繁荣和福利以及保障国防的使命，旨在开展研究，以创建一个支持从大量、多维、时间序列观测数据中生成、评估和改进特设模型的基础设施。特设模型创建中的挑战来自于模型实现方式的组合爆炸性数量。该框架Synapse旨在支持和简化海量流数据的自然迭代和交互式模型构建过程。建模者只需要指定一组基本的引导参数;框架将管理与以下相关的复杂性：（1）如何分散流，（2）如何管理数据访问，（3）应对I/O和内存争用，以及（4）模型生成工作负载的分散。该研究涉及可扩展的数据分散技术，采用分布式哈希表数据结构，基于map-reduce的工作流和模型创建工作负载的编排，训练数据管理和模型性能的交互式可视化评估。可视化组件将使建模人员能够快速有效地评估多种模型的质量，每种模型可能覆盖输入特征空间的不同部分，并使用这些评估来指导关于模型选择、更新或替换的决策。如果成功，该框架将随着数据量、可用数据流数量、模型生成工作负载和实时模型评估的增加而扩展。