Tensor Network Representation for Machine Learning: Theoretical Study and Algorithms Development

我们开发了几种新的张量网络分解和完备化算法，并开发了基于张量网络的神经网络模型和学习算法。这些方法已经被应用到多个计算机视觉任务中，具体来说，我们开发了张量化的RNN模型，可以实现长期记忆和减少模型大小;我们还研究了贝叶斯潜在因子模型，以了解张量网络如何能够实现模型压缩;我们提出的张量融合层可以应用于具有改进性能的图像表示任务，这也可以应用于多模态情感分析的开发。 从理论上研究了张量核范数，提出了几种新的张量范数定义，为张量的精确恢复提供了保证。 此外，我们还提出了一种新的张量网络，称为全连接张量网络，它在模拟张量模式之间的复杂相互作用方面表现出很大的灵活性。我们的理论和模型的有效性得到了广泛的验证张量完成任务。

We have developed several new tensor network decomposition and completion algorithms, and also developed the tensor network based neural network models and learning algorithms. These methods have been applied to several computer vision tasks.Specifically, we have developed tensorized RNN model that can achieve long term memory and reduced model size; we also studied Bayesian latent factor models to understand how tensor network is able to achieve model compression; our proposed tensor fusion layer can be applied to image denoting tasks with improvement performance, which can be also applied to the development of multimodal sentimental analysis. We also developed an efficient algorithm for classification on incomplete data samples, which has practical applications when the high-quality dataset is difficult to be obtained.From theoretical perspective, we have studied tensor nuclear norm and proposed several new definition of tensor norm, which has guarantee for exact recovery to tensor. In addition, we proposed a new type tensor network, called fully connected tensor network, which shows great flexibility on modeling complex interaction between tensor modes. The effectiveness of our theory and model is validated extensively on tensor completion tasks.