Applications of category theory and topology to machine learning

点击翻译按钮获取中文摘要

Topology is the branch of mathematics that studies shapes and knotting phenomena and quantifies holes. Over the past decade it has contributed a library of important tools in data science. Category theory is the branch of mathematics that provides unifying language for relationships and logical structures across many domains. There are 3 concrete components to this project:1. In this component of the work, we relate recurrent neural network models to categorical structures and universal algebra. Recurrent models are the cornerstone of modern time series analysis and natural language processing. This work component is being carried out in collaboration with a small AI tech firm called Hylomorphism. We will describe a class of models in terms of categorical structures called anamorphisms and catamorphisms and then relate the resulting models to more mainstream recurrent neural network based models. 2. We will explore neural networks from the perspective of tropical geometry. Expanding on recent work of Yue Ren, we will study new initialization schemes for neural networks based on tropical geometry and how these can improve the quality and efficiency of the training process. The current standard initialization scheme is to use a uniform or Gaussian distribution for the weights. Tropical geometry shows that there is potential to improve on this by taking into account the polyhedral structure of objects associated with the network. By adjusting initial weights to avoid geometrically tricky points, training via gradient descent can proceed more smoothly.3. We will explore diffusion-based methods for approximating the calculation of persistent homology and other invariants from topological data analysis. Diffusion maps are an incredibly popular and powerful technique for dimensional reduction and approximate clustering, Persistent homology in dimension zero also offers a kind of approximate clustering. The first step will be to explore the relation between these. Then we will move to higher dimensions.