Lower Dimensional Representation of Text Data for Efficient and Effective Information Retrieval

摘要由于当今互联网和计算能力的指数级增长，信息检索系统有望处理海量数据，用户需要更高效的技术来从海量数据中获取有用的信息。本研究的目标是在基于向量空间的信息检索中找到文本数据的低维表示。为了高效有效地处理当今信息检索系统中的海量数据，降维势在必行。这里一个基本的重要问题是用相对严重的降维来实现数据的更好表示，而不是通过矩阵的较低等级近似来实现简单的降维。一个困难是，用理论公式来衡量某种降维方法对原始数据的良好表示程度并不容易，必须在进行实验研究的同时进行理论研究。

ABSTRACT 0204109Haesun ParkUniversity of Minnesota- Twin CitiesDue to today's exponential growth of the internet and computing power, an information retrievalsystem is expected to handle a tremendous amount of data, and users demand more efficient techniquesto obtain useful information from the flood of data. The goal of this proposed research is to find lower dimensional representations of text data in vector space based information retrieval. Dimension reduction is imperative for achieving high efficiency and effectiveness in manipulating the massive quantity of data in today's information retrieval system. A problem of fundamental importance here is to achieve better representation of the data with relatively severe dimension reduction, rather than simple dimension reduction through a lower rank approximation of a matrix. One difficulty is that it is not easy to measure by a theoretical formula how well a certain dimension reduction method provides a good representation of the original data, and it will be essential to conduct theoretical research in parallel with experimental study.