Tensor Network Representation for Machine Learning: Theoretical Study and Algorithms Development

我们提出了几种新的张量网络分解和完备化算法，并提出了基于张量网络的神经网络模型和学习算法。这些方法已经应用于多个计算机视觉任务中。具体地说，我们开发了能够实现长时间记忆和降低模型规模的张量RNN模型；我们还研究了贝叶斯潜在因子模型，以了解张量网络如何实现模型压缩；我们提出的张量融合层可以用于性能更好的图像表示任务，也可以应用于多模式情感分析的开发。从理论上对张量核范数进行了研究，提出了几种新的张量范数定义，保证了张量的精确恢复。此外，我们还提出了一种新型的张量网络，称为完全连通张量网络，它在模拟张量模式之间的复杂相互作用方面表现出了很大的灵活性。我们的理论和模型在张量补全任务上得到了广泛的验证。

We have developed several new tensor network decomposition and completion algorithms, and also developed the tensor network based neural network models and learning algorithms. These methods have been applied to several computer vision tasks.Specifically, we have developed tensorized RNN model that can achieve long term memory and reduced model size; we also studied Bayesian latent factor models to understand how tensor network is able to achieve model compression; our proposed tensor fusion layer can be applied to image denoting tasks with improvement performance, which can be also applied to the development of multimodal sentimental analysis. We also developed an efficient algorithm for classification on incomplete data samples, which has practical applications when the high-quality dataset is difficult to be obtained.From theoretical perspective, we have studied tensor nuclear norm and proposed several new definition of tensor norm, which has guarantee for exact recovery to tensor. In addition, we proposed a new type tensor network, called fully connected tensor network, which shows great flexibility on modeling complex interaction between tensor modes. The effectiveness of our theory and model is validated extensively on tensor completion tasks.