Information Processing Theory and Applications

赖斯大学：约翰逊，DON h .密切相关的信号处理和信息论领域从未找到共同点。信号处理的重点是信号如何表示信息以及系统如何操纵和改变信号结构。信息论围绕着信号所代表的信息结构展开，但通过专注于有效的压缩和通信而忽略了对接收者有意义的信息。该研究开发了一种新的信息处理理论，将这两个学科结合在一起，具有双重目标：理解信号（无论其性质如何）如何有效地表示信息，以及量化系统处理信息的能力。由于该理论的通用性，我们分析了通信系统和神经处理系统，以了解它们如何有效地传达信息和意义，以及神经群如何处理和表示信息。我们通过计算与编码信息的两个实例相关联的信号之间的信息论距离（它遵循数据处理定理）来量化信号表示信息的程度。我们假设信号是随机的，距离测量了与信号相关的概率分布的不同程度。我们使用Kullback-Leibler距离，因为它通过Stein引理与最优分类器性能和通过Cramer-Rao界与最优最小二乘估计器性能相关。因此，更大的距离对应于更有效的信息表示。一个系统的信息处理能力是通过信息传递比来衡量的，信息传递比被定义为在系统输入和输出处计算的距离之比。有了这个比率，我们就可以量化信息处理系统作为信息过滤器的表现。

PROPOSAL #0105558WILLIAM MARSH RICE UNIVERSITYPI: JOHNSON, DON H.The closely allied fields of signal processing and information theory have never found common ground. Signal processing focuses on how signals can represent information and how systems manipulate and change signal structure. Information theory revolves around the structure of information that signals represent, but ignores what information is meaningful to the receiver by concentrating on efficient compression and communication. The research develops a new theory of information processing that weds these two disciplines and has the dual goals of understanding how effectively signals, no matter what their nature, can represent information and of quantifying how well systems process information. Because of the theory's generality, we analyze both communication systems, to probe how effectively they convey information and meaning, and neural processing systems, to understand how neural groups process and represent information.We quantify how well signals represent information by computing an information-theoretic distance (it obeys the Data Processing Theorem) between signals associated with two instances of the encoded information. We assume that the signals are stochastic, and the distance measures how different are the probability distributions associated with the signals. We use the Kullback-Leibler distance because it is related both to optimal classifier performance via Stein's Lemma and to optimal least-squares estimator performance through the Cramer-Rao bound. A larger distance thus corresponds to a more effective representation of the information. The information processing ability of a system is measured by the information transfer ratio, defined to be the ratio of distances computed at the system's input and output. With this ratio, we quantify how well an information processing system behaves as an information filter.