BIGDATA: Collaborative Research: F: Stochastic Approximation for Subspace and Multiview Representation Learning

对有用特征或表示的无监督学习是机器学习最基本的挑战之一。无监督表示学习技术利用未标记的数据，这些数据通常是廉价和丰富的，有时几乎是无限的。这些无处不在的技术的目标是学习一种揭示数据中固有的低维结构的表示，通过结合普遍的人工智能先验(如平滑和稀疏性)来理清潜在的变异因素，并对多个任务和领域有用。这个项目的目的是开发新的理论和方法来表示学习，可以很容易地扩展到大型数据集。特别是，本项目涉及大规模无监督特征学习的方法，包括主成分分析(PCA)和偏最小二乘(PLS)。为了利用大量未标记的数据，该项目将开发适当的计算方法，并在数据加载的情况下进行研究。政权。因此，与将表征学习视为降维技术并关注有限数据上的经验目标不同，研究这些方法的目的是基于样本来优化总体目标。这一观点建议使用随机近似方法，如随机梯度下降(SGD)和随机镜像下降，这些方法本质上是增量的，并通过计算成本较低的更新来处理每个新样本。此外，这一观点使我们能够严格分析随机近似算法相对于传统有限数据方法的优势。该项目旨在开发用于主成分分析和偏最小二乘的随机逼近方法以及相关问题和扩展，包括深度和稀疏变量，并在数据加载的情况下分析这些问题。

Unsupervised learning of useful features, or representations, is one of the most basic challenges of machine learning. Unsupervised representation learning techniques capitalize on unlabeled data which is often cheap and abundant and sometimes virtually unlimited. The goal of these ubiquitous techniques is to learn a representation that reveals intrinsic low-dimensional structure in data, disentangles underlying factors of variation by incorporating universal AI priors such as smoothness and sparsity, and is useful across multiple tasks and domains. This project aims to develop new theory and methods for representation learning that can easily scale to large datasets. In particular, this project is concerned with methods for large-scale unsupervised feature learning, including Principal Component Analysis (PCA) and Partial Least Squares (PLS). To capitalize on massive amounts of unlabeled data, this project will develop appropriate computational approaches and study them in the ?data laden? regime. Therefore, instead of viewing representation learning as dimensionality reduction techniques and focusing on an empirical objective on finite data, these methods are studied with the goal of optimizing a population objective based on sample. This view suggests using Stochastic Approximation approaches, such as Stochastic Gradient Descent (SGD) and Stochastic Mirror Descent, that are incremental in nature and process each new sample with a computationally cheap update. Furthermore, this view enables a rigorous analysis of benefits of stochastic approximation algorithms over traditional finite-data methods. The project aims to develop stochastic approximation approaches to PCA and PLS and related problems and extensions, including deep, and sparse variants, and analyze these problems in the data-laden regime.