BIGDATA: Collaborative Research: F: Stochastic Approximation for Subspace and Multiview Representation Learning

无监督学习有用的特征或表示是机器学习最基本的挑战之一。无监督表示学习技术利用无标记数据，这些数据通常便宜且丰富，有时几乎是无限的。这些无处不在的技术的目标是学习一种表示，这种表示可以揭示数据中固有的低维结构，通过结合平滑性和稀疏性等通用人工智能先验来解决潜在的变化因素，并且在多个任务和领域中都很有用。该项目旨在为表示学习开发新的理论和方法，可以很容易地扩展到大型数据集。特别地，本项目关注大规模无监督特征学习的方法，包括主成分分析（PCA）和偏最小二乘（PLS）。为了利用大量未标记的数据，本项目将开发适当的计算方法，并在？数据拉登?政权。因此，代替将表示学习视为降维技术和专注于有限数据上的经验目标，这些方法的研究目标是基于样本优化总体目标。这种观点建议使用随机逼近方法，如随机梯度下降（SGD）和随机镜像下降，它们本质上是增量的，并且通过计算廉价的更新来处理每个新样本。此外，这种观点使我们能够严格分析随机近似算法相对于传统有限数据方法的优势。该项目旨在开发PCA和PLS及其相关问题和扩展的随机逼近方法，包括深度和稀疏变体，并在数据负载状态下分析这些问题。

Unsupervised learning of useful features, or representations, is one of the most basic challenges of machine learning. Unsupervised representation learning techniques capitalize on unlabeled data which is often cheap and abundant and sometimes virtually unlimited. The goal of these ubiquitous techniques is to learn a representation that reveals intrinsic low-dimensional structure in data, disentangles underlying factors of variation by incorporating universal AI priors such as smoothness and sparsity, and is useful across multiple tasks and domains. This project aims to develop new theory and methods for representation learning that can easily scale to large datasets. In particular, this project is concerned with methods for large-scale unsupervised feature learning, including Principal Component Analysis (PCA) and Partial Least Squares (PLS). To capitalize on massive amounts of unlabeled data, this project will develop appropriate computational approaches and study them in the ?data laden? regime. Therefore, instead of viewing representation learning as dimensionality reduction techniques and focusing on an empirical objective on finite data, these methods are studied with the goal of optimizing a population objective based on sample. This view suggests using Stochastic Approximation approaches, such as Stochastic Gradient Descent (SGD) and Stochastic Mirror Descent, that are incremental in nature and process each new sample with a computationally cheap update. Furthermore, this view enables a rigorous analysis of benefits of stochastic approximation algorithms over traditional finite-data methods. The project aims to develop stochastic approximation approaches to PCA and PLS and related problems and extensions, including deep, and sparse variants, and analyze these problems in the data-laden regime.