BIGDATA: Collaborative Research: F: Stochastic Approximation for Subspace and Multiview Representation Learning

有用特征或表示的无监督学习是机器学习最基本的挑战之一。无监督表示学习技术利用未标记的数据，这些数据通常是廉价和丰富的，有时几乎是无限的。这些无处不在的技术的目标是学习一种表示，揭示数据中的内在低维结构，通过结合通用AI先验（如平滑性和稀疏性）来解开潜在的变化因素，并在多个任务和领域中有用。该项目旨在开发新的理论和方法，用于表示学习，可以轻松扩展到大型数据集。特别是，该项目涉及大规模无监督特征学习的方法，包括主成分分析（PCA）和偏最小二乘（PLS）。为了利用大量的未标记的数据，该项目将开发适当的计算方法，并在？数据负载？政权因此，这些方法不是将表示学习视为降维技术并专注于有限数据上的经验目标，而是以基于样本优化群体目标为目标进行研究。这种观点建议使用随机近似方法，如随机梯度下降（SGD）和随机镜像下降，这些方法本质上是增量的，并且使用计算成本较低的更新来处理每个新样本。此外，这一观点使得严格的分析的好处随机逼近算法在传统的有限数据方法。该项目旨在开发PCA和PLS的随机近似方法以及相关问题和扩展，包括深度和稀疏变量，并在数据负载制度中分析这些问题。

Unsupervised learning of useful features, or representations, is one of the most basic challenges of machine learning. Unsupervised representation learning techniques capitalize on unlabeled data which is often cheap and abundant and sometimes virtually unlimited. The goal of these ubiquitous techniques is to learn a representation that reveals intrinsic low-dimensional structure in data, disentangles underlying factors of variation by incorporating universal AI priors such as smoothness and sparsity, and is useful across multiple tasks and domains. This project aims to develop new theory and methods for representation learning that can easily scale to large datasets. In particular, this project is concerned with methods for large-scale unsupervised feature learning, including Principal Component Analysis (PCA) and Partial Least Squares (PLS). To capitalize on massive amounts of unlabeled data, this project will develop appropriate computational approaches and study them in the ?data laden? regime. Therefore, instead of viewing representation learning as dimensionality reduction techniques and focusing on an empirical objective on finite data, these methods are studied with the goal of optimizing a population objective based on sample. This view suggests using Stochastic Approximation approaches, such as Stochastic Gradient Descent (SGD) and Stochastic Mirror Descent, that are incremental in nature and process each new sample with a computationally cheap update. Furthermore, this view enables a rigorous analysis of benefits of stochastic approximation algorithms over traditional finite-data methods. The project aims to develop stochastic approximation approaches to PCA and PLS and related problems and extensions, including deep, and sparse variants, and analyze these problems in the data-laden regime.