III: Small: Collaborative Research: Scalable Deep Bayesian Tensor Decomposition

现实世界中的许多应用，例如在线购物、推荐、社交媒体和信息安全，都涉及不同实体之间的交互。例如，网上购物行为可以简单地用顾客、商品和购物网站之间的交互来描述。这些相互作用自然地由张量表示，张量是多维数组。每个维度代表一种实体类型（例如，客户或商品），每个元素描述特定的交互（例如，购买/未购买）。该项目旨在开发灵活高效的张量分解方法，能够发现张量中实体之间的各种复杂关系，处理实际应用中的海量数据，适应数据的快速增长。所开发的方法可用于促进许多重要的预测和知识发现任务，例如提高推荐准确性、预测广告点击率、了解错误信息如何通过社交媒体传播以及检测恶意手机应用程序。尽管现有的张量分解方法取得了成功，但它们使用多线性分解形式或浅核，并且无法捕获数据中高度复杂的关系。然而，由于实际应用的多样性和复杂性，复杂和非线性的关系、效应和模式无处不在。此外，缺乏高效、可扩展的非线性分解算法来处理当今规模空前的静态张量和快速且持续增长的动态张量。该项目旨在开发可扩展的深度贝叶斯张量分解方法，最大限度地提高捕获各种复杂关系的灵活性，以前所未有的规模和快速数据流有效处理静态数据，并为嵌入估计和预测提供不确定性量化。该研究将通过以下方式完成：（1）设计新的贝叶斯张量分解模型，其中包含深度架构以提高估计复杂函数的能力，（2）开发去中心化的异步学习算法来处理超大规模静态张量，（3）开发在线增量学习算法来处理快速数据流并在接收新数据时产生响应更新，而无需从头开始重新训练，以及（4）对合成和现实世界大数据进行综合评估。所提出的研究将贡献一个显着改进的张量分解工具集，该工具集能够强大地估计任意复杂的关系，可扩展到前所未有的规模（例如数十亿个节点和数万亿个条目）的静态张量，并通过高效的增量更新实现快速数据流。此外，随着贝叶斯方法的发展，该工具集对噪声具有弹性，为不确定性量化提供后验分布，并将所有可能的结果整合到稳健的预测中。一旦工具集可用，对张量高阶关系的理解以及对相关模式（例如社区和异常）的挖掘将得到极大的增强；社交链接、点击率和推荐等兴趣量化的预测性能将得到显着提升。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，认为值得支持。

Many applications in the real world, such as online shopping, recommendation, social media and information security, involve interactions among different entities. For example, online shopping behaviors can be simply described by the interactions between customers, commodities and shopping web sites. These interactions are naturally represented by tensors, which are arrays of multiple dimensions. Each dimension represents a type of entities (e.g., customers or commodities), and each element describes a particular interaction (e.g, purchased/not purchased). The project aims to develop flexible and efficient tensor decomposition approaches that can discover a variety of complicated relationships between the entities in tensors, handle a tremendous amount of data from practical applications, and adapt to rapid data growth. The developed approaches can be used to promote many important prediction and knowledge discovery tasks, such as improving the recommendation accuracy, predicting advertisement click rates, understanding how misinformation propagation through social media, and detecting malicious cell- phone apps. Despite the success of the existing tensor decomposition approaches, they use multilinear decomposition forms or shallow kernels, and are incapable of capturing highly complicated relationships in data. However, complex and nonlinear relationships, effects and patterns are ubiquitous, due to the diversity and complexity of the practical applications. Furthermore, there is a lack of efficient, scalable nonlinear decomposition algorithms to handle static tensors nowadays at unprecedented scales, and dynamic tensors that grow fast and continuously. The project aims to develop scalable deep Bayesian tensor decomposition approaches that maximize the flexibility to capture all kinds of complex relationships, efficiently process static data at unprecedented scales and rapid data streams, and provide uncertainty quantification for both embedding estimations and predictions. The research will be accomplished through: (1) the design of new Bayesian tensor decomposition models that incorporate deep architectures to improve the capability of estimating intricate functions, (2) the development of decentralized, asynchronous learning algorithms to process extremely large-scale static tensors, (3) the development of online incremental learning algorithms to handle rapid data streams and to produce responsive updates upon receiving new data, without retraining from scratch, and (4) comprehensive evaluations on both synthetic and real-world big data. The proposed research will contribute a markedly improved tensor decomposition toolset that are powerful to estimate arbitrarily complex relationships, scalable to static tensors at unprecedented scales (e.g., billions of nodes and trillions of entries) and to fast data streams with efficient incremental updates. Moreover, as Bayesian approaches, the toolset are resilient to noise, provide posterior distributions for uncertainty quantification, and integrate all possible outcomes into robust predictions. Once the toolsets are available, the understanding of the high-order relationships in tensors, and the mining of associated patterns, such as communities and anomalies, will be enormously enhanced; the predictive performance for the quantify of interests, such as social links, click-through-rates, and recommendation, will be dramatically promoted.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.