Representation learning and exploration of data geometries

在过去的二十年里，快速的技术进步彻底改变了几乎每个科学技术领域中大数据的可用性。现代数据以越来越大的数量产生，通过多次测量进行量化，并经常以许多批次或样本的形式收集，这些批次或样本编码了重要的差异(例如，在不同的观测环境、收集时间或收集技术之间)。原始观测数据的这种可用性为新发现提供了潜力，达到了以前从未可能达到的水平。然而，它也带来了许多数据分析挑战，需要新一代机器学习工具来支持对数据的无监督探索，并增强领域专家从海量数据集中提取新知识和见解的能力，而不是自动对其进行注释。拟议的研究计划直接与人们对探索性数据分析工具日益增长的兴趣联系在一起，这些工具提取简化的数据表示，用于推断或揭示紧急模式、结构和动态。其中一个重要的重点是提供易于处理、人类可解释的表示法，使数据可用于研究、假设生成和领域专家的进一步探索，这些专家不一定是面向计算的。具体地说，这项建议将侧重于通过利用流形学习、图形信号处理和最近新兴的几何深度学习领域的工具来构建这种表示。虽然许多数据密集型领域是相关的，需要这样的数据探索工具，但拟议的研究预计将对两个应用领域产生直接和直接的影响。首先，单细胞数据分析为理解高分辨率生物异质性带来了令人振奋的新前景，但由于收集的数据的批次效应、高维和稀疏性，也带来了各种挑战。此外，单细胞数据的探索性处理通常需要将固有数据几何与数据分布分开，因为稀有子群体和亚稳定状态之间的稀疏转换通常是生物医学数据分析中非常感兴趣的，但如果分析的重点是主要的分布模式，这些就会丢失。其次，为了了解深层神经网络令人印象深刻的机器学习能力，对深层神经网络中潜在表征的研究最近变得流行起来。事实上，作为线性运算和相对简单的非线性的级联，神经网络自然地提供了以任务为导向的内部表示，由隐藏层中的神经元激活给出。在这里，探索性数据分析可以帮助理解随着神经网络专门处理不同任务而出现的渐进信息处理机制，并阐明数据分布对神经元激活空间(例如，激活流形)的内在结构的影响。

In the past two decades, rapid technological advances have revolutionized the availability of big high dimensional data in virtually every field of science and technology. Modern data are being produced at increasingly large volumes, quantified by numerous measurements, and often collected in many batches or samples that encode nontrivial variations (e.g., between different observation settings, collection times, or collection technologies). Such availability of raw observational data provides potential for new discoveries at a level that was never possible before. However, it also introduces numerous data analysis challenges that require a new generation of machine learning tools to enable unsupervised exploration of data and enhance the ability of domain experts to extract new knowledge and insights from massive datasets, rather than automate their ability to annotate them. The proposed research program directly ties into the growing interest in exploratory data analysis tools that extract simplified data representations for inferring or uncovering emergent patterns, structures, and dynamics. A strong emphasis in these is on providing tractable, human-interpretable, representations that make data approachable for study, hypothesis generation, and further exploration by domain experts who are not necessarily computation oriented. Concretely, this proposal will focus on constructing such representations by leveraging tools from manifold learning, graph signal processing, and the recently emerging field of geometric deep learning. While numerous data-intensive fields are relevant and in need of such data exploration tools, the proposed research is expected to have immediate and direct impact on two application fields. First, single cell data analysis introduces exciting new prospects for understanding high resolution biological heterogeneity, but also gives rise to various challenges due to batch effects, high dimenstionality, and sparsity of the collected data. Furthermore, exploratory processing of single cell data often requires separation of intrinsic data geometry from data distribution as rare subpopulations and sparse transitions between meta-stable states are often of great interest in biomedical data analysis, but they would be lost by an analysis that focuses on the main distribution modes. Second, the study of latent representations in deep neural networks has recently gained popularity in order to understand their impressive machine learning capabilities. Indeed, as cascades of linear operations and relatively-simple nonlinearities, neural networks naturally provide task-oriented internal representations given by neuron activations in hidden layers. Here, exploratory data analysis can help understand the gradual information processing mechanisms that emerge as neural nets specialize on varied tasks and shed light on the effects of data distribution on the intrinsic structure of neuron activation spaces (e.g., activation manifolds).