Information Processing Theory and Applications

信息处理理论与应用

• 批准号: 0105558
• 负责人: Don Johnson
• 金额: --
• 依托单位: William Marsh Rice University
• 依托单位国家: 美国
• 项目类别: Continuing grant
• 财政年份: 2001
• 资助国家: 美国
• 起止时间: 2001-07-01 至 2004-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0105558&HistoricalAwards=false
• 关键词: Information; Processing; Theory; Applications

PROPOSAL #0105558WILLIAM MARSH RICE UNIVERSITYPI: JOHNSON, DON H.The closely allied fields of signal processing and information theory have never found common ground. Signal processing focuses on how signals can represent information and how systems manipulate and change signal structure. Information theory revolves around the structure of information that signals represent, but ignores what information is meaningful to the receiver by concentrating on efficient compression and communication. The research develops a new theory of information processing that weds these two disciplines and has the dual goals of understanding how effectively signals, no matter what their nature, can represent information and of quantifying how well systems process information. Because of the theory's generality, we analyze both communication systems, to probe how effectively they convey information and meaning, and neural processing systems, to understand how neural groups process and represent information.We quantify how well signals represent information by computing an information-theoretic distance (it obeys the Data Processing Theorem) between signals associated with two instances of the encoded information. We assume that the signals are stochastic, and the distance measures how different are the probability distributions associated with the signals. We use the Kullback-Leibler distance because it is related both to optimal classifier performance via Stein's Lemma and to optimal least-squares estimator performance through the Cramer-Rao bound. A larger distance thus corresponds to a more effective representation of the information. The information processing ability of a system is measured by the information transfer ratio, defined to be the ratio of distances computed at the system's input and output. With this ratio, we quantify how well an information processing system behaves as an information filter.

提案#0105558威廉·马什·赖斯大学：约翰逊，唐·H。信号处理和信息理论的密切相关领域从未找到共同点。信号处理专注于信号如何表示信息以及系统如何操纵和改变信号结构。信息论围绕着信号所代表的信息结构，但通过专注于有效的压缩和通信，忽略了什么信息对接收者有意义。该研究开发了一种新的信息处理理论，将这两个学科结合起来，并具有双重目标，即理解信号如何有效地表示信息，无论其性质如何，以及量化系统处理信息的能力。 由于该理论的通用性，我们分析了两个通信系统，以探索它们如何有效地传递信息和意义，以及神经处理系统，以了解神经群如何处理和表示信息。我们通过计算与编码信息的两个实例相关联的信号之间的信息理论距离（它遵守数据处理定理）来量化信号表示信息的程度。 我们假设信号是随机的，距离度量与信号相关的概率分布的差异。 我们使用Kullback-Leibler距离，因为它与通过Stein引理的最佳分类器性能和通过Cramer-Rao界的最佳最小二乘估计器性能有关。 因此，更大的距离对应于更有效的信息表示。系统的信息处理能力是通过信息传递率来衡量的，信息传递率定义为在系统的输入和输出处计算的距离的比率。有了这个比率，我们量化了信息处理系统作为信息过滤器的表现。