A Study on Advanced Network Information Filtering System

HTTP代理服务器是在服务器和客户端之间传输HTTP消息的应用层网关，对于在广域网上高效地传输HTTP消息起着重要的作用。本研究旨在研究增强现有HTTP代理服务器系统“高级网络信息过滤”功能的基本技术，项目的第一年重点是设计和实现“通用过滤服务器”(CF服务器)，它与现有的HTTP代理服务器协作执行中转消息的内容转换。在CF服务器的设计中，我们采用具有独特内容转换机制的通用应用协议网关Delegate作为协作代理服务器，并利用HTTP消息头模板库扩展了Delegate的CFI功能。我们的CF服务器是使用脚本语言Perl实现的，以确认与流行的HTTP客户端的交互。未来的问题包括CF服务器和Delegate之间的协作操作，CF服务器的加速，以及应用实例的考虑。项目的第二年专注于对HTTP访问日志的分析。HTTP访问日志可能包含丰富的信息，可用于协作缓存中缓存服务器的分配或服务器配对。然而，由于数据量庞大，传统方法难以进行分析。为了解决这一问题，采用了最近在信息检索中使用的一种自动索引技术LSI(潜在语义索引)，并对大规模的HTTP日志文件进行了相似度评估。预处理阶段采用Kullback-Leibler信息准则，利用URL层次结构对海量日志数据进行汇总。使用实际的HTTP日志数据演示了汇总的效果。

HTTP proxy servers, the application level gateways which transfer HTTP messages between servers and clients, play an important role in efficient transmission of HTTP messages over wide area networks. This research aims at studying basic technologies for enhancing the current HTTP proxy servers systems with "advanced network information filtering" functions.The first year of the project focuses on the design and implementation of 'Common Filter Server' (CF server) which performs contents conversion of transit messages in cooperation with existing HTTP proxy servers. In our design of CF server, we assume DeleGate, a general-purpose application protocol gateway with unique contents conversion mechanism called CFI (Common Filter Interface), as cooperating proxy server and extend CFI function of DeleGate using a database of HTTP message header templates. Our CF server is implemented using script language Perl to confirm the interaction with popular HTTP clients. Future issues include cooperative operation between CF server and DeleGate, the speed up of the CF server, and consideration of application examples.The second year of the project focuses on the analysis of HTTP access logs. HTTP access log possibly contains rich information useful for cache server allocation or pairings of servers in collaborative caching. However, conventional methods suffer intractability of the analysis due to the hugeness of the data. In order to deal with the problem, LSI (Latent Semantic Indexing), an automatic indexing technique recently used in information retrieval, is adopted to large-scale HTTP log files and estimates the similarity between two Internet domains. Kullback-Leibler information criteria is applied in the pre-processing stage to summarize huge amount of log data utilizing the URL hierarchy. The effect of the summarization is demonstrated using actual HTTP log data.