Applications of category theory and topology to machine learning

拓扑学是数学的一个分支，它研究形状和打结现象，并对孔洞进行量化。在过去的十年里，它为数据科学贡献了一个重要工具库。范畴论是数学的一个分支，它为许多领域的关系和逻辑结构提供了统一的语言。该项目有三个具体组成部分：1。在这部分工作中，我们将递归神经网络模型与分类结构和泛代数联系起来。递归模型是现代时间序列分析和自然语言处理的基石。这项工作是与一家名为Hylomorphism的小型人工智能技术公司合作进行的。我们将描述一类模型的分类结构称为anamorphisms和catamorphisms，然后将所得模型与更主流的基于递归神经网络的模型相关联。2.我们将从热带几何的角度来探索神经网络。在Yue Ren最近工作的基础上，我们将研究基于热带几何的神经网络的新初始化方案，以及这些方案如何提高训练过程的质量和效率。当前的标准初始化方案是使用权重的均匀或高斯分布。热带几何表明，通过考虑与网络相关的物体的多面体结构，有可能改进这一点。通过调整初始权重以避免几何上的棘手点，通过梯度下降进行训练可以更顺利地进行。3.我们将探索基于扩散的方法，用于近似计算拓扑数据分析中的持久同源性和其他不变量。扩散映射是一种非常流行和强大的降维和近似聚类技术，零维持久同调也提供了一种近似聚类。第一步将是探索它们之间的关系。然后我们将进入更高的维度。

Topology is the branch of mathematics that studies shapes and knotting phenomena and quantifies holes. Over the past decade it has contributed a library of important tools in data science. Category theory is the branch of mathematics that provides unifying language for relationships and logical structures across many domains. There are 3 concrete components to this project:1. In this component of the work, we relate recurrent neural network models to categorical structures and universal algebra. Recurrent models are the cornerstone of modern time series analysis and natural language processing. This work component is being carried out in collaboration with a small AI tech firm called Hylomorphism. We will describe a class of models in terms of categorical structures called anamorphisms and catamorphisms and then relate the resulting models to more mainstream recurrent neural network based models. 2. We will explore neural networks from the perspective of tropical geometry. Expanding on recent work of Yue Ren, we will study new initialization schemes for neural networks based on tropical geometry and how these can improve the quality and efficiency of the training process. The current standard initialization scheme is to use a uniform or Gaussian distribution for the weights. Tropical geometry shows that there is potential to improve on this by taking into account the polyhedral structure of objects associated with the network. By adjusting initial weights to avoid geometrically tricky points, training via gradient descent can proceed more smoothly.3. We will explore diffusion-based methods for approximating the calculation of persistent homology and other invariants from topological data analysis. Diffusion maps are an incredibly popular and powerful technique for dimensional reduction and approximate clustering, Persistent homology in dimension zero also offers a kind of approximate clustering. The first step will be to explore the relation between these. Then we will move to higher dimensions.