CiC (RDDC): Wordsmith in the Cloud - Refining Language Models Using Web-Scale Language Networks

网络用户在网络社区中表达自己从未如此容易。从在线对话中产生的文本信息的过载是各种消费者群体所关注的。许多有效的工具来管理这种信息过载，如文本检索系统，依赖于使用统计语言模型。然而，语言模型的质量受到数据稀疏、上下文不匹配和无法建模语义关系的限制。在这个项目中，云计算系统，如Azure和Hadoop和新的分布式算法将被用来从Web规模的文本集合中提取异构语言网络。这些语言网络将用于平滑和上下文化各个领域的语言模型，使其准确和健壮。改进的语言模型将有助于改进最先进的文本检索和挖掘技术，增强跨社区和语言界限的真实的用户的信息访问和知识获取体验。技术和资源（例如，语言网络和精炼的语言模型）将使分析社交媒体和许多其他领域中的文本内容的广泛用户受益。该项目的研究成果，包括软件工具和资源，将在项目网站上公布（http：//www-personal.umich.edu/wangqmei/wordsmith/）。

It has never been easier for Web users to express themselves in online communities. The overload of text information generated from online conversations is of concern to various consumer groups. Many effective tools to manage this information overload, such as text retrieval systems, rely on the use of statistical language models. The quality of language models is however limited by the sparseness of data, the mismatch of context, and the incapability of modeling semantic relations. In this project, cloud computing systems like Azure and Hadoop and novel distributed algorithms will be employed to extract heterogeneous language networks from Web-scale text collections. These language networks will be used to smooth and contextualize language models in various domains, making them accurate and robust. The refined language models will help improve state-of-the-art text retrieval and mining techniques, enhancing the information access and knowledge acquisition experience of real users across community and language boundaries. The techniques and resources (e.g., language networks and refined language models) will benefit a broad range of users that analyze text content in social media and many other domains. Research results of this project, including software tools and resources, will be published on the project Web site (http://www-personal.umich.edu/~qmei/wordsmith/).