Unsupervised Document Set Exploration Using Divisive Partitioning

这个项目的目的是开发算法和工具，用于探索和分类巨大的文档体，特别是来自万维网的文档。该技术方法基于一种新的分层分裂划分方法，在初步测试中快速生成了高质量的聚类。要解决的研究问题包括：可伸缩性分析、理论基础、增量更新方法、一般化（例如处理缺失值和不同的伸缩）以及为各种应用程序与一个或多个Web代理的接口。鉴于其跨学科的性质，教育研讨会和教程是该项目的自然组成部分。预期的结果是一组用于组织大型文档集合的算法和工具，这些算法和工具具有以下特点：(1)可扩展到非常大的数据集，(2)无监督操作，以及(3)所发现类别的合理质量和有用性。预期的好处包括数据集规模的数量级增加，在此基础上以无监督的方式提取有用的类别将是可行的。潜在的应用程序包括客户端WWW组织和搜索辅助，以一致的方式创建文档评级的服务器端辅助，维护和更新专门数据库内容的组织和分类的工具，所有这些都需要最少的人工干预。http://www.cs.umn.edu/~boley/PDDP.html

The purpose of this project is to develop algorithms and tools for the exploration and categorization of extremely large bodies of documents, especially from the World Wide Web. The technical approach is based on a new hierarchical divisive partitioning method which has produced quality clusters very fast in preliminary tests. The research issues to be addressed include: scalability analysis, theoretical foundations, incremental updating methods, generalizations (such as handling missing values and different scaling), and interface to one or more Web agents for various applications. Educational seminars and tutorials are a natural part of this project, given its interdisciplinary nature. Anticipated results are a set of algorithms and tools for organizing large document collections that enjoy the features of (1) scalability to very large datasets, (2) unsupervised operation, and (3) reasonable quality and usefulness of the categories found. Anticipated benefits include an order of magnitude increase in the size of datasets on which it will be practical to extract useful categories in an unsupervised manner. Potential applications include client-side WWW organization and search aids, server-side aids to create document ratings in a consistent manner, tools to maintain and update organization and classification of contents of specialized databases, all with a minimum of human intervention. http://www.cs.umn.edu/~boley/PDDP.html