III: Small: Collaborative Research: Scalable Deep Bayesian Tensor Decomposition

III：小：协作研究：可扩展的深贝叶斯张量分解

• 批准号: 1910983
• 负责人: Shandian Zhe
• 金额: $ 29.84万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910983&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Scalable

Many applications in the real world, such as online shopping, recommendation, social media and information security, involve interactions among different entities. For example, online shopping behaviors can be simply described by the interactions between customers, commodities and shopping web sites. These interactions are naturally represented by tensors, which are arrays of multiple dimensions. Each dimension represents a type of entities (e.g., customers or commodities), and each element describes a particular interaction (e.g, purchased/not purchased). The project aims to develop flexible and efficient tensor decomposition approaches that can discover a variety of complicated relationships between the entities in tensors, handle a tremendous amount of data from practical applications, and adapt to rapid data growth. The developed approaches can be used to promote many important prediction and knowledge discovery tasks, such as improving the recommendation accuracy, predicting advertisement click rates, understanding how misinformation propagation through social media, and detecting malicious cell- phone apps. Despite the success of the existing tensor decomposition approaches, they use multilinear decomposition forms or shallow kernels, and are incapable of capturing highly complicated relationships in data. However, complex and nonlinear relationships, effects and patterns are ubiquitous, due to the diversity and complexity of the practical applications. Furthermore, there is a lack of efficient, scalable nonlinear decomposition algorithms to handle static tensors nowadays at unprecedented scales, and dynamic tensors that grow fast and continuously. The project aims to develop scalable deep Bayesian tensor decomposition approaches that maximize the flexibility to capture all kinds of complex relationships, efficiently process static data at unprecedented scales and rapid data streams, and provide uncertainty quantification for both embedding estimations and predictions. The research will be accomplished through: (1) the design of new Bayesian tensor decomposition models that incorporate deep architectures to improve the capability of estimating intricate functions, (2) the development of decentralized, asynchronous learning algorithms to process extremely large-scale static tensors, (3) the development of online incremental learning algorithms to handle rapid data streams and to produce responsive updates upon receiving new data, without retraining from scratch, and (4) comprehensive evaluations on both synthetic and real-world big data. The proposed research will contribute a markedly improved tensor decomposition toolset that are powerful to estimate arbitrarily complex relationships, scalable to static tensors at unprecedented scales (e.g., billions of nodes and trillions of entries) and to fast data streams with efficient incremental updates. Moreover, as Bayesian approaches, the toolset are resilient to noise, provide posterior distributions for uncertainty quantification, and integrate all possible outcomes into robust predictions. Once the toolsets are available, the understanding of the high-order relationships in tensors, and the mining of associated patterns, such as communities and anomalies, will be enormously enhanced; the predictive performance for the quantify of interests, such as social links, click-through-rates, and recommendation, will be dramatically promoted.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

真实的世界中的许多应用，如在线购物、推荐、社交媒体和信息安全，都涉及不同实体之间的交互。例如，网上购物行为可以简单地用顾客、商品和购物网站之间的交互来描述。这些相互作用自然地由张量表示，张量是多维数组。每个维度表示一种类型的实体（例如，顾客或商品），并且每个元素描述特定的交互（例如，购买/未购买）。该项目旨在开发灵活高效的张量分解方法，可以发现张量中实体之间的各种复杂关系，处理来自实际应用的大量数据，并适应快速的数据增长。所开发的方法可以用于促进许多重要的预测和知识发现任务，例如提高推荐准确率，预测广告点击率，了解错误信息如何通过社交媒体传播，以及检测恶意手机应用程序。尽管现有的张量分解方法取得了成功，但它们使用多线性分解形式或浅核，并且无法捕获数据中高度复杂的关系。然而，由于实际应用的多样性和复杂性，复杂和非线性的关系，效果和模式是普遍存在的。此外，目前缺乏有效的、可扩展的非线性分解算法来处理前所未有规模的静态张量和快速连续增长的动态张量。该项目旨在开发可扩展的深度贝叶斯张量分解方法，最大限度地提高捕获各种复杂关系的灵活性，以前所未有的规模和快速数据流有效处理静态数据，并为嵌入估计和预测提供不确定性量化。研究将通过以下方式完成：（1）设计新的贝叶斯张量分解模型，该模型包含深度架构以提高估计复杂函数的能力，（2）开发分散的异步学习算法以处理超大规模静态张量，（3）开发在线增量学习算法以处理快速数据流并在接收新数据时产生响应更新，无需从头开始重新培训，以及（4）对合成和真实世界的大数据进行全面评估。拟议的研究将贡献一个显着改进的张量分解工具集，该工具集能够强大地估计任意复杂的关系，可扩展到前所未有规模的静态张量（例如，数十亿个节点和数万亿个条目），并通过有效的增量更新来快速数据流。此外，随着贝叶斯方法的发展，工具集对噪声具有弹性，为不确定性量化提供后验分布，并将所有可能的结果整合到稳健的预测中。一旦工具集可用，对张量中高阶关系的理解以及对相关模式（如社区和异常）的挖掘将大大增强;对兴趣的量化的预测性能，诸如社交链接、点击率和推荐，该奖项反映了NSF的法定使命，并被认为是值得通过评估使用基金会的智力价值和更广泛的支持。影响审查标准。

项目成果

Bayesian Continuous-Time Tucker Decomposition

• 发表时间: 2022
• 作者: Shikai Fang;A. Narayan;Robert M. Kirby;Shandian Zhe

Scalable Gaussian Process Regression Networks

可扩展的高斯过程回归网络

• 发表时间: 2020
• 期刊: International Joint Conference on Artificial Intelligence - Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI
• 作者: Li, Shibo Li;Xing, Wei;Kirby, Robert M.;Zhe, Shandian
• 通讯作者: Zhe, Shandian

Multi-Fidelity High-Order Gaussian Processes for Physical Simulation

• 发表时间: 2020-06
• 期刊: ArXiv
• 作者: Zheng Wang;Wei W. Xing;R. Kirby;Shandian Zhe

Probabilistic Neural-Kernel Tensor Decomposition

• DOI: 10.1109/icdm50108.2020.00062
• 发表时间: 2020-11
• 期刊: 2020 IEEE International Conference on Data Mining (ICDM)
• 作者: Conor Tillinghast;Shikai Fang;Kai Zhang;Shandian Zhe

Streaming Nonlinear Bayesian Tensor Decomposition

• 发表时间: 2020
• 作者: Zhimeng Pan;Z. Wang;Shandian Zhe