Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
基本信息
- 批准号:RGPIN-2019-06776
- 负责人:
- 金额:$ 2.04万
- 依托单位:
- 依托单位国家:加拿大
- 项目类别:Discovery Grants Program - Individual
- 财政年份:2022
- 资助国家:加拿大
- 起止时间:2022-01-01 至 2023-12-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Modern computer systems consist of multiple components that coordinate their input, output and processing to provide timely information to their human or machine users. These components form distributed systems with processing and data transmission possible at every node. One source of system input has been the Internet of Things, with the massive deployment of sensors (including images and video). A typical deployment scenario is the growing market of wearable devices in fitness/health monitoring. Individual devices can generate large amounts of data over short time periods, processed both locally and by a centralized organization. These body-worn devices may or may not have continuous connection to the Internet for appropriate cost-effective data transmission. Delivery protocols are required that prioritize data requirements and are resource-efficient. Some monitoring data must, however, be transferred in real-time. Similar data streams will be generated by home automation and industrial sensors, such as mining, manufacturing, and agriculture. They, too require delivery of large amounts of data at different volumes, rates and latency requirements, constituting streams of big data. When data arrives at a centralized processing location, another problem is created. The total aggregate of data from many sources (individuals, industrial sensors, security cameras and smartphones) overwhelms the capacities of the largest computers available. Parallel processing techniques and frameworks developed by organizations such as Google, Yahoo! and Facebook for Big Data processing have been the preferred solutions for the last dozen years or so. As new frameworks are introduced, their applicability in emerging domains, such as environmental sensor networks and wearable networks requires configuration, tuning and parameter settings. This will require a mixture of manual resource allocation and scheduling policies. The main research goal investigated through this program is to identify and quantify the performance/resource utilization tradeoff in distributed systems consisting of data collection from multiple diverse sources that require aggregate data processing to find common patterns and characteristics. I aim to build techniques to identify configurations that achieve preferred, cost-effective deployments that meet user goals with regards to this tradeoff. Various strategies for reducing resource requirements (such as energy/communication bandwidth/disk space) result in a loss of quality of information or the rate at which the information can be provided. Appropriate adjustments in scheduling, routing or resource allocation between the various components will enable this degradation in service to be mitigated. The outcomes will be methods of evaluating the tradeoffs, case studies using these evaluation methods and development of prototype systems that implement adaptive strategies fo collecting and processing sensor network big data.
现代计算机系统由多个组件组成,这些组件协调它们的输入、输出和处理,以向其人类或机器用户提供及时的信息。这些组件形成了分布式系统,每个节点都可以进行处理和数据传输。随着传感器(包括图像和视频)的大规模部署,物联网一直是系统输入的一个来源。一个典型的部署场景是健身/健康监测中可穿戴设备的市场不断增长。单个设备可以在短时间内生成大量数据,这些数据既可以在本地处理,也可以由中央组织处理。这些穿戴在身上的设备可能会或可能不会连续连接到互联网,以实现适当的经济高效的数据传输。传输协议需要确定数据需求的优先顺序,并且资源高效。然而,一些监测数据必须实时传输。类似的数据流将由家庭自动化和工业传感器产生,如采矿、制造业和农业。它们也需要以不同的容量、速率和延迟要求交付大量数据,这构成了大数据流。当数据到达集中处理位置时,就会产生另一个问题。来自许多来源(个人、工业传感器、安全摄像头和智能手机)的数据的总聚合超出了现有最大计算机的能力。由Google、Yahoo!等组织开发的并行处理技术和框架和Facebook的大数据处理一直是过去十几年左右的首选解决方案。随着新框架的引入,它们在新兴领域的适用性,如环境传感器网络和可穿戴网络,需要配置、调整和参数设置。这将需要混合使用手动资源分配和调度策略。通过该计划调查的主要研究目标是识别和量化分布式系统中的性能/资源利用权衡,该系统包括从多个不同来源收集的数据,这些数据需要聚合数据处理以找到共同的模式和特征。我的目标是构建一些技术,以确定实现首选的、经济高效的部署的配置,以满足用户在此权衡方面的目标。减少资源需求(例如能量/通信带宽/磁盘空间)的各种策略会导致信息质量或提供信息的速率的损失。在不同组件之间的调度、路由或资源分配方面的适当调整将使这种服务降级能够得到缓解。其成果将是评估权衡的方法、使用这些评估方法的案例研究以及实施收集和处理传感器网络大数据的自适应策略的原型系统的开发。
项目成果
期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
数据更新时间:{{ journalArticles.updateTime }}
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Makaroff, Dwight其他文献
Makaroff, Dwight的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Makaroff, Dwight', 18)}}的其他基金
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2021
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2020
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2019
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Packet processing on CPU and FPGA using software-defined networking
使用软件定义网络在 CPU 和 FPGA 上进行数据包处理
- 批准号:
505304-2016 - 财政年份:2016
- 资助金额:
$ 2.04万 - 项目类别:
Engage Plus Grants Program
Network Packet Processing with P4 (Programming Protocol-Independent Packet Processors) in CPU, GPU and FPGAs using OpenCL
使用 OpenCL 在 CPU、GPU 和 FPGA 中使用 P4(独立于编程协议的数据包处理器)进行网络数据包处理
- 批准号:
486746-2015 - 财政年份:2015
- 资助金额:
$ 2.04万 - 项目类别:
Engage Grants Program
Performance and security in parallel and distributed systems
并行和分布式系统的性能和安全性
- 批准号:
227765-2011 - 财政年份:2015
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Performance and security in parallel and distributed systems
并行和分布式系统的性能和安全性
- 批准号:
227765-2011 - 财政年份:2014
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Real-time communication performance for mobile devices
移动设备的实时通信性能
- 批准号:
446427-2013 - 财政年份:2013
- 资助金额:
$ 2.04万 - 项目类别:
Engage Grants Program
Performance and security in parallel and distributed systems
并行和分布式系统的性能和安全性
- 批准号:
227765-2011 - 财政年份:2013
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Performance and security in parallel and distributed systems
并行和分布式系统的性能和安全性
- 批准号:
227765-2011 - 财政年份:2012
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
相似国自然基金
Graphon mean field games with partial observation and application to failure detection in distributed systems
- 批准号:
- 批准年份:2025
- 资助金额:0.0 万元
- 项目类别:省市级项目
基于“阳化气、阴成形”理论探讨龟鹿二仙胶调控 HIF-1α/Systems Xc-通路抑制铁死亡治疗少弱精子症的作用机理
- 批准号:
- 批准年份:2024
- 资助金额:15.0 万元
- 项目类别:省市级项目
EstimatingLarge Demand Systems with MachineLearning Techniques
- 批准号:
- 批准年份:2024
- 资助金额:万元
- 项目类别:外国学者研究基金
Understanding complicated gravitational physics by simple two-shell systems
- 批准号:12005059
- 批准年份:2020
- 资助金额:24.0 万元
- 项目类别:青年科学基金项目
Simulation and certification of the ground state of many-body systems on quantum simulators
- 批准号:
- 批准年份:2020
- 资助金额:40 万元
- 项目类别:
全基因组系统作图(systems mapping)研究三种细菌种间互作遗传机制
- 批准号:31971398
- 批准年份:2019
- 资助金额:58.0 万元
- 项目类别:面上项目
The formation and evolution of planetary systems in dense star clusters
- 批准号:11043007
- 批准年份:2010
- 资助金额:10.0 万元
- 项目类别:专项基金项目
相似海外基金
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2021
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Travel: Travel Support for The 42nd IEEE International Conference on Distributed Computing Systems (ICDCS 2022)
差旅:第 42 届 IEEE 国际分布式计算系统会议 (ICDCS 2022) 差旅支持
- 批准号:
2139584 - 财政年份:2021
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2020
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Algorithmic Support for Massive Scale Distributed Systems
大规模分布式系统的算法支持
- 批准号:
EP/T01461X/1 - 财政年份:2020
- 资助金额:
$ 2.04万 - 项目类别:
Research Grant
Travel Support for the 39th IEEE International Conference on Distributed Computing Systems (ICDCS 19)
第 39 届 IEEE 国际分布式计算系统会议 (ICDCS 19) 的差旅支持
- 批准号:
1931341 - 财政年份:2019
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant
Distributed Systems Support for Processing Big Data from Sensor Networks
分布式系统支持处理来自传感器网络的大数据
- 批准号:
RGPIN-2019-06776 - 财政年份:2019
- 资助金额:
$ 2.04万 - 项目类别:
Discovery Grants Program - Individual
Travel Support for the 38th IEEE International Conference on Distributed Computing Systems (ICDCS 18)
第 38 届 IEEE 国际分布式计算系统会议 (ICDCS 18) 的差旅支持
- 批准号:
1836366 - 财政年份:2018
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant
Travel Support for the 37th IEEE International Conference on Distributed Computing Systems (ICDCS 17)
第 37 届 IEEE 国际分布式计算系统会议 (ICDCS 17) 的差旅支持
- 批准号:
1742939 - 财政年份:2017
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant
Travel Support for the 36th IEEE International Conference on Distributed Computing Systems (ICDCS 16)
第 36 届 IEEE 国际分布式计算系统会议 (ICDCS 16) 的差旅支持
- 批准号:
1639580 - 财政年份:2016
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant
Travel Support for the 35th IEEE International Conference on Distributed Computing Systems (ICDCS 2015)
第 35 届 IEEE 国际分布式计算系统会议 (ICDCS 2015) 差旅支持
- 批准号:
1540768 - 财政年份:2015
- 资助金额:
$ 2.04万 - 项目类别:
Standard Grant