Large Systems and Big Data: Models, Tools, Analysis, and Algorithms

大型系统和大数据：模型、工具、分析和算法

• 批准号: RGPIN-2020-04075
• 负责人: Mazumdar, Ravi
• 金额: $ 2.84万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=751628
• 关键词: Large; Systems; Big; Data; Models

Understanding the behaviour of complex networked systems as clouds, CRANs, and social networks is critical as their deployments continue to grow rapidly. Another important class of problems is the development of methodologies to understand dependencies between different data sets so that we can obtain better forecasts and understand the temporal flow of information. The proposed research will have applications to weather prediction, financial markets, and medical imaging. The proposed research will consist of model building, the development of analytical tools, and the design of efficient algorithms to achieve the goals. The research addresses challenges related to operating and delivering real-time services on clouds, distributed servers and in 5G systems, and ways of detecting time dependencies in information flow to provide better predictive power. Of interest is the understanding of how to build distributed network architectures to deliver low latency performance and parsimonious prediction models. The research will provide better understanding on the design of load balancing in clouds and information retrieval when systems are in high load and understanding other load balancing strategies such as redundancy type mechanisms in large systems. Drawing on methodologies from stochastic models, randomized algorithms, convex optimization, graph approximation, and high-- dimensional statistics the research will focus on: 1) The development of low latency algorithms for load balancing and information retrieval in large distributed architectures such as clouds or distributed caching systems, 2) Understanding the information dynamics and consensus among a large group of interacting agents with social network applications and problems of the spread of infection, and 3) Understanding the temporal dependence in time series data from different sources with the aim of developing parsimonious representations for prediction algorithms. The first two themes are characterized by scale in the numbers of interacting entities and while the third involves inference of high dimensional statistics from big data. Success in this research will have important scientific and technological contributions. On the scientific side we will obtain new understanding on good latency policies and   extend mean-field techniques to a larger class of models. A better understanding of redundancy models will also be of use in other important applications such as organ exchange networks and hospital surgery scheduling. Understanding the dynamics of information flow in random networks will allow us to design better algorithms for faster information flow that can be critical in both social network and military applications. Finally obtaining better predictive models to deal with time series data is crucial in a wide variety of applications arising in IoT, medical imaging, finance, econometric models, and weather prediction.

随着云、CRANs和社交网络等复杂网络系统的部署持续快速增长，理解它们的行为至关重要。另一类重要的问题是开发方法来理解不同数据集之间的依赖关系，以便我们能够获得更好的预测并理解信息的时间流。拟议中的研究将应用于天气预报、金融市场和医学成像。建议的研究将包括建立模型，开发分析工具，以及设计有效的算法来实现目标。该研究解决了与在云、分布式服务器和5G系统上运行和提供实时服务相关的挑战，以及检测信息流中的时间依赖性以提供更好预测能力的方法。我们感兴趣的是理解如何构建分布式网络架构以提供低延迟性能和简洁的预测模型。该研究将有助于更好地理解云环境中负载平衡的设计和系统高负载时的信息检索，以及大型系统中冗余类型机制等其他负载平衡策略。利用随机模型、随机算法、凸优化、图近似和高维统计的方法，研究将集中在：1)在大型分布式架构（如云或分布式缓存系统）中开发用于负载平衡和信息检索的低延迟算法；2)理解与社交网络应用程序交互的大型代理之间的信息动态和共识以及感染传播问题；3)了解来自不同来源的时间序列数据的时间依赖性，目的是为预测算法开发简洁的表示。前两个主题的特点是相互作用实体数量的规模，而第三个主题涉及从大数据中推断高维统计数据。这项研究的成功将有重要的科学和技术贡献。在科学方面，我们将获得对良好延迟策略的新理解，并将平均场技术扩展到更大的模型类别。更好地理解冗余模型也将用于器官交换网络和医院手术调度等其他重要应用。了解随机网络中信息流的动态将使我们能够设计出更好的算法，以实现更快的信息流，这在社交网络和军事应用中都是至关重要的。最后，获得更好的预测模型来处理时间序列数据对于物联网、医疗成像、金融、计量经济模型和天气预报等各种应用至关重要。