Microtext Processing

术语“微文本”描述了一种非常短的文本文档类型(通常是一句话或更少)，以非正式的方式书写并且未经编辑以保证质量，并且包括一些元数据。微文本在当今世界已经变得无处不在，包括在线用户评论、Facebook新闻提要和推文以及搜索查询。就其本质而言，微文本与传统的自然语言处理(NLP)是非常不同的挑战。它们的篇幅太短，阻碍了依赖词频和统计数据的自然语言处理方法，而大量的拼写错误、打字错误和非常规缩略语使标准语言资源无济于事。微文本处理应该被视为自然语言处理的一个新的分支。这项提议结合了四个平行的五年微文本研究计划。第一个目标是开发微文本理解算法，专门用于使用公司知识库的查询理解和回答任务。第二个目标是开发微文本理解算法，专门用于在Twitter上非常精确的事件检测任务。第三个目标是开发专门用于挖掘微博讨论的微文本理解算法，特别是在检索软件开发信息的背景下。第四个目标是开发一种“军事沃森”，它可以监控军事聊天，并向军官通报发生的关键国际事件。所有四个项目都是与外部研究和行业合作伙伴合作完成的。此外，它们都符合一个统一的长期目标：开拓和发展微文本处理的新领域，创造微文本专业知识的核心。微文本处理的发展将是语义网和Web2.0发展的下一个重要里程碑。今天产生的微文本数量绝对是巨大的：例如，谷歌上有47亿次网络查询，每天有3.4亿条推文发布。网络用户积极寻找和接受新的微文本技术。这项研究对加拿大经济的战略重要性怎么强调都不为过，我们的多家合作伙伴公司和组织将直接受益于我们的结果。

The term "microtext" describes a type of text document that is very short (typically one sentence or less), is written in an informal manner and unedited for quality, and includes some metadata. Microtexts have become omnipresent in today's world, and include online user comments, Facebook newsfeeds and Tweets, and search queries. By their very nature, microtexts are a very different challenge from traditional Natural Language Processing (NLP). Their short length hinders NLP methodologies that rely on word frequencies and statistics, and the abundance of spelling mistakes, typos, and unconventional abbreviations make standard linguistic resources unhelpful. Microtext processing should be seen as a new branch of NLP.This proposal brings together four parallel five-year microtext research programs. The first aims to develop microtext understanding algorithms specialized to the task of query understanding and answering using corporate knowledge bases. The second aims to develop microtext understanding algorithms specialized to the task of very precise event detection in Twitter. The third aims to develop microtext understanding algorithms specialized to the task of mining microblog discussions, specifically in the context of retrieving software development information. And the fourth aims to develop a "military Watson" that can monitor military chat and inform officers of crucial international events occurring. All four projects are done in collaboration with external research and industrial partners. Moreover, they all feed into a single unifying long-term objective: to pioneer and develop the new field of microtext processing, and create a core of microtext expertise. The development of microtext processing will be the next major milestone in Semantic Web and Web 2.0 development. The amount of microtext generated today is absolutely massive: for example, there are 4.7 billion web queries asked on Google and 340 million Tweets posted every day. Web users actively seek out and embrace new microtext technology. The strategic importance of this research for the Canadian economy cannot be overstated, and our multiple partner companies and organizations will benefit directly from our results.