Applications of category theory and topology to machine learning

拓扑学是研究形状和打结现象以及量化孔洞的数学分支。在过去的十年中，它为数据科学贡献了一个重要的工具库。范畴论是数学的一个分支，它为跨多个领域的关系和逻辑结构提供了统一的语言。该项目有三个具体组成部分：1。在这部分工作中，我们将递归神经网络模型与分类结构和通用代数联系起来。循环模型是现代时间序列分析和自然语言处理的基石。这项工作是与一家名为Hylomorphism的小型人工智能技术公司合作进行的。我们将根据分类结构描述一类模型，称为变形和变形，然后将所得模型与更主流的基于循环神经网络的模型联系起来。2. 我们将从热带几何的角度探讨神经网络。在岳仁最近工作的基础上，我们将研究基于热带几何的神经网络的新初始化方案，以及这些方案如何提高训练过程的质量和效率。当前的标准初始化方案是使用均匀或高斯分布的权重。热带几何表明，考虑到与网络相关的物体的多面体结构，有可能改进这一点。通过调整初始权重来避免几何上棘手的点，通过梯度下降的训练可以进行得更顺利。我们将探索基于扩散的方法来逼近拓扑数据分析中持久同调和其他不变量的计算。扩散映射是一种非常流行和强大的降维和近似聚类技术，零维的持久同调也提供了一种近似聚类。第一步将是探索它们之间的关系。然后我们将移动到更高的维度。

Topology is the branch of mathematics that studies shapes and knotting phenomena and quantifies holes. Over the past decade it has contributed a library of important tools in data science. Category theory is the branch of mathematics that provides unifying language for relationships and logical structures across many domains. There are 3 concrete components to this project:1. In this component of the work, we relate recurrent neural network models to categorical structures and universal algebra. Recurrent models are the cornerstone of modern time series analysis and natural language processing. This work component is being carried out in collaboration with a small AI tech firm called Hylomorphism. We will describe a class of models in terms of categorical structures called anamorphisms and catamorphisms and then relate the resulting models to more mainstream recurrent neural network based models. 2. We will explore neural networks from the perspective of tropical geometry. Expanding on recent work of Yue Ren, we will study new initialization schemes for neural networks based on tropical geometry and how these can improve the quality and efficiency of the training process. The current standard initialization scheme is to use a uniform or Gaussian distribution for the weights. Tropical geometry shows that there is potential to improve on this by taking into account the polyhedral structure of objects associated with the network. By adjusting initial weights to avoid geometrically tricky points, training via gradient descent can proceed more smoothly.3. We will explore diffusion-based methods for approximating the calculation of persistent homology and other invariants from topological data analysis. Diffusion maps are an incredibly popular and powerful technique for dimensional reduction and approximate clustering, Persistent homology in dimension zero also offers a kind of approximate clustering. The first step will be to explore the relation between these. Then we will move to higher dimensions.