Microtext Processing

点击翻译按钮获取中文摘要

The term "microtext" describes a type of text document that is very short (typically one sentence or less), is written in an informal manner and unedited for quality, and includes some metadata. Microtexts have become omnipresent in today's world, and include online user comments, Facebook newsfeeds and Tweets, and search queries. By their very nature, microtexts are a very different challenge from traditional Natural Language Processing (NLP). Their short length hinders NLP methodologies that rely on word frequencies and statistics, and the abundance of spelling mistakes, typos, and unconventional abbreviations make standard linguistic resources unhelpful. Microtext processing should be seen as a new branch of NLP. This proposal brings together four parallel five-year microtext research programs. The first aims to develop microtext understanding algorithms specialized to the task of query understanding and answering using corporate knowledge bases. The second aims to develop microtext understanding algorithms specialized to the task of very precise event detection in Twitter. The third aims to develop microtext understanding algorithms specialized to the task of mining microblog discussions, specifically in the context of retrieving software development information. And the fourth aims to develop a "military Watson" that can monitor military chat and inform officers of crucial international events occurring. All four projects are done in collaboration with external research and industrial partners. Moreover, they all feed into a single unifying long-term objective: to pioneer and develop the new field of microtext processing, and create a core of microtext expertise. The development of microtext processing will be the next major milestone in Semantic Web and Web 2.0 development. The amount of microtext generated today is absolutely massive: for example, there are 4.7 billion web queries asked on Google and 340 million Tweets posted every day. Web users actively seek out and embrace new microtext technology. The strategic importance of this research for the Canadian economy cannot be overstated, and our multiple partner companies and organizations will benefit directly from our results.