Metric-based mutual information estimators and their application to analysing neural recordings Abstract

基于度量的互信息估计器及其在分析神经记录中的应用摘要

• 批准号: 2445980
• 金额: --
• 依托单位: University of Bristol
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2445980
• 关键词: Metric; based; mutual; information; estimators

Since Shannon's introduction of Information Theory, it has been applied widely and has found great success in a wide range of fields. These fields range from data compression through linguistics, to thermodynamics. It allows the quantification of the information contained in distributions of random variables, and how these distributions relate to one another. These are fundamental, widely important ideas - explaining its use and popularity. One key measure in Information Theory is mutual information. This captures the amount of information learning one variable gives you about another. This measure has become popular in areas such as machine learning, and is growing in popularity. However, reliably, quickly calculating or estimating this quantity can prove challenging. In practice, this often involves calculating the whole distribution, followed by logarithmic and other non linear quantities, which can be hard to estimate, biased and slow to converge. In (Houghton, 2015) (Houghton, 2019), a new approach to estimating mutual information was introduced. This approximator relies on embedding data in a metric space, and exploiting the distance measure in that space. It is a Kozachenko-Leonenko estimator, estimating the mutual information from the relationship between data points, rather than from the points themselves. This gives the effect that it works well on high dimensional data, as it is this distance used in the estimator, not the data points. Practically, making use of the distance metric means that this estimator can work with a much smaller data set than traditional or naive methods. The effects and uses of this method will be explored, focussing on analysing neural recordings, and in machine learning. Mutual information, or information gain, is a common quantity in machine learning and AI, either implicitly or explicitly. Explicit examples include splitting data sets to construct decision trees, and for measuring information flow in neural networks. Implicitly, as mutual information captures the amount of shared information in two random variables, machine learning and inference problems can often be framed using Information Theory and mutual information. One of the areas Information Theory has been very useful in is neuroscience. One of the primary functions of the brain is to process and store information, so applying Information Theoretic approaches is an obvious fit. Experiments in neuroscience are, as data acquisition and experimental techniques improve, giving a lot of temporal high dimensional data, and often data of different types. The model independent nature of Information Theory means it is able to capture a very wide range of phenomena and interactions, as they are not limited by assuming a model. An integral part of this estimator is having a 'sensible' distance measure on the space of spike train recordings, a problem for which multiple measures have been proposed and will be investigated for use in estimating mutual information. This gives an area that this kind of estimator is suited to; one where large datasets can be costly or unfeasible. We aim to apply this mutual information estimator to biological data, recorded in Matt Jones' lab. The aim here would be, through the estimator, to work out the relationships between recorded neurons, and infer something of the network. Similar approaches have been used before, though in a more restricted setting, to good results, and we explore the effect of this new approach. Overall, this project will study and apply this estimator, in the context of neuroscience on recorded data, as well as in the context of machine learning and for artificial neural networks - and to develop the theory of mutual information estimators. This research falls within the EPSRC Artificial Intelligence Technologies research area. References Houghton, C. (2015). Calculating mutual information for spike trains and other data with distances but no coordinates.

自香农提出信息论以来，信息论得到了广泛的应用，并在各个领域取得了巨大的成功。这些领域从数据压缩到语言学，再到热力学。它允许量化随机变量分布中包含的信息，以及这些分布如何相互关联。这些都是基本的、广泛重要的想法--解释了它的用途和受欢迎程度。信息论中的一个关键衡量标准是互信息。这捕获了一个变量向您提供的关于另一个变量的信息量。这一措施已经在机器学习等领域变得流行起来，并越来越受欢迎。然而，可靠、快速地计算或估计这一数量可能会被证明是具有挑战性的。在实践中，这通常涉及计算整个分布，然后是对数和其他非线性量，这些量可能很难估计、有偏差并且收敛速度很慢。在(Houghton，2015)(Houghton，2019)中，引入了一种新的估计互信息的方法。这种近似器依赖于在度量空间中嵌入数据，并利用该空间中的距离度量。它是一种Kozachenko-Leonenko估计器，通过数据点之间的关系来估计互信息，而不是从点本身。这给出了它在高维数据上工作得很好的效果，因为它是在估计器中使用的距离，而不是数据点。实际上，使用距离度量意味着该估计器可以处理比传统或幼稚方法小得多的数据集。我们将探索这种方法的效果和用途，重点放在分析神经记录和机器学习上。互信息，或信息增益，是机器学习和人工智能中的一个共同的量，无论是隐含的还是显式的。明确的例子包括拆分数据集以构建决策树，以及测量神经网络中的信息流。隐含地，由于互信息捕捉两个随机变量中共享的信息量，机器学习和推理问题通常可以用信息论和互信息来框架。信息论在神经科学中非常有用的领域之一。大脑的主要功能之一是处理和存储信息，因此应用信息论方法显然是合适的。神经科学中的实验，随着数据采集和实验技术的改进，提供了大量的时态高维数据，而且往往是不同类型的数据。信息论与模型无关的本质意味着它能够捕捉到非常广泛的现象和相互作用，因为它们不受假设模型的限制。这个估计器的一个组成部分是在尖峰列车记录的空间上有一个“合理的”距离测量，这个问题已经被提出了多种测量方法，并将被研究用于估计互信息。这给出了一个这种估计器适合的领域；在这个领域，大型数据集可能代价高昂，或者不可行。我们的目标是将这种互信息估计器应用于马特·琼斯实验室记录的生物数据。这里的目的是，通过估计器，找出记录的神经元之间的关系，并推断出网络的一些东西。类似的方法以前也使用过，尽管在更有限的环境中，取得了良好的结果，我们探索了这种新方法的效果。总体而言，本项目将在记录数据的神经科学背景下，以及在机器学习和人工神经网络的背景下，研究和应用这一估计器--并发展互信息估计器理论。这项研究属于EPSRC人工智能技术研究领域。参考文献Houghton，C.(2015)。计算有距离但没有坐标的尖峰列车和其他数据的互信息。