III: Small: Collaborative Research: Cost-Efficient Sampling and Estimation from Large-Scale Networks

III：小型：协作研究：大规模网络的经济高效采样和估计

• 批准号: 1908375
• 负责人: Chul-Ho Lee
• 金额: $ 25万
• 依托单位: Florida Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2022-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908375&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Cost

Sampling and estimating structural information from large-scale networks or graphs has been central to our understanding of the network dynamics and its rich set of applications. Markov Chain Monte Carlo (MCMC) has been the key enabler for a broader context of graph sampling, including estimating the properties of large graphs, sampling the corpus of documents indexed by search engines, sampling records from hidden databases behind Web forms, identifying subgraphs of certain characteristics and frequent graph pattern matching. Despite versatile applications of the MCMC methods and their customized algorithms for analyzing graph-structured data in various forms, there still exist critical challenges and limitations in the literature centered around the MCMC methods. One is the 'cost' consumption/constraints associated with the sampling operation, which limits the size of total samples obtained and negatively affects the accuracy of any estimator based on the obtained samples. Another limitation is that the recent advances in MCMC, especially built up on favorable non-reversible Markov chains, cannot be leveraged to the various large-graph sampling tasks, due to their required global knowledge of the underlying state space, lack of distribution implementation, unconstrained state space, as well as the simplified cost assumption. The goal of this research is to fully exploit the potentials of a set of crawling samplers by making the samplers adaptive and possibly interactive on a properly constructed graph domain, to transcend the current status-quo in the wide range of graph sampling tasks. Specifically, the project aims to: (i) build a theoretical framework to construct a suite of cost-efficient sampling policies by optimally balancing the tradeoff between the sample quality and quantity under challenged access environments with a given cost budget, (ii) design a class of adaptive random walks by fully exploiting the past information to achieve minimal temporal correlations over the obtained samples and by controlling the random walks collectively to enable maximal space exploration, and (iii) extend the standard MCMC toolkits toward faster and more cost-efficient exploration of feasible subgraphs/configurations and computing/optimization on a graph, along with extensive validations to create practical and usable solutions in reality. This research has a high potential impact on a vast range of multi-disciplinary applications, including sampling large-scale graphs for statistical inference and efficient estimation and randomized algorithms for combinatorial optimizations in various disciplines, where the standard MCMC methods have been dominant but also constrained our understanding.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从大规模网络或图中采样和估计结构信息是我们理解网络动力学及其丰富应用的核心。马尔可夫链蒙特卡罗（Markov Chain Monte Carlo， MCMC）已经成为更广泛的图采样上下文的关键推动者，包括估计大型图的属性、对搜索引擎索引的文档语库进行采样、从Web表单后面的隐藏数据库中采样记录、识别某些特征的子图以及频繁的图模式匹配。尽管MCMC方法及其自定义算法可用于分析各种形式的图结构数据，但在以MCMC方法为中心的文献中仍然存在关键的挑战和局限性。一个是与采样操作相关的“成本”消耗/约束，它限制了获得的总样本的大小，并对基于获得的样本的任何估计器的准确性产生负面影响。另一个限制是MCMC的最新进展，特别是建立在有利的不可逆马尔可夫链上的MCMC，由于它们需要对底层状态空间的全局知识，缺乏分布实现，无约束状态空间以及简化的成本假设，不能用于各种大图采样任务。本研究的目标是充分利用一组爬行采样器的潜力，使采样器在适当构建的图域上自适应和可能的交互，以超越目前在广泛的图采样任务中的现状。具体而言，该项目旨在：(i)在给定成本预算的受限访问环境下，通过最优地平衡样本质量和数量之间的权衡，构建一个理论框架，以构建一套具有成本效益的采样策略；（ii）通过充分利用过去的信息，设计一类自适应随机漫步，以实现所获得样本的最小时间相关性，并通过控制随机漫步来实现最大的空间探索；（iii）扩展标准MCMC工具包，以便更快、更经济地探索可行的子图/配置和图上的计算/优化，以及在现实中创建实用和可用的解决方案的广泛验证。这项研究对广泛的多学科应用具有很高的潜在影响，包括采样大规模图用于统计推断和高效估计，以及随机算法用于各种学科的组合优化，其中标准MCMC方法一直占主导地位，但也限制了我们的理解。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

An Efficient and Scalable Algorithm for Estimating Kemeny's Constant of a Markov Chain on Large Graphs

• DOI: 10.1145/3447548.3467431
• 发表时间: 2021-08
• 期刊: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
• 作者: Shiju Li;Xin Huang;Chul-Ho Lee

Estimating Distributions of Large Graphs from Incomplete Sampled Data

• DOI: 10.23919/ifipnetworking52078.2021.9472848
• 发表时间: 2021-06
• 期刊: 2021 IFIP Networking Conference (IFIP Networking)
• 作者: Shiju Li;Xin Huang;Chul-Ho Lee

Trapping Malicious Crawlers in Social Networks

• DOI: 10.1145/3340531.3412004
• 发表时间: 2020-10
• 期刊: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
• 作者: Shiju Li;Chul-Ho Lee;Do Young Eun