String-based and Unification-based Methodology for Text mining and Processing

在当今时代，信息技术对经济和社会的影响可能在更广泛意义上的所谓社交媒体和互联网领域是最直接和最显著的。向网络贡献可访问内容的人的比例已经从很小的一小部分变成了很大一部分，很可能是大多数。已经确定的是，一方面，这股大数据洪流可以增加我们的知识和效率，带来更大的社会效益，但另一方面，我们在错过相关信息的同时，经常迷失和淹没在无关的信息中。移动技术在任何地方都可以使用，但在屏幕大小、用户输入速度和处理能力方面也受到限制，这对信息检索、查询和相关信息的引导提出了更高的要求。*这里，我们建议开发三种不同的核心自然语言处理方法，为解决这一信息管理问题做出强有力的贡献。除了理论结果之外，我们还开发了几个实际实现所设计解决方案的工具，并将这些方法和工具应用于特定的应用领域。*我们的方法可以根据语言处理的级别分为三个层次：(1)公共N-Gram分析(CNG)，(2)基于正则表达式和有限状态处理(RegEx)，(3)基于统一的类型化特征结构处理和匹配(Unif)，目的是协调这些技术。N元语法模型、正则表达式和基于统一的语法在NLP中是很好理解的。我们方法的新颖性在于在这些模型的基础上开发了更具体的方法：新的n元语法分析和距离函数，以及可视化；迭代正则表达式替换，以及基于随机统一的匹配和子图同构方法。

The economic and social impact of information technology in the current time is probably most direct and significant in the area of the so-called social media and Internet in a wider sense. The ratio of people contributing accessible content to the Web has gone from a minor fraction to a large portion, likely a majority. It has been established that this flood of "big data" on one side can increase our knowledge and efficiency, leading to a greater social benefit, but on another side we are frequently lost and drowning in irrelevant information while missing relevant information. The proliferation of mobile technology, which is available anywhere but also limited in screen size, user input rate and processing power, has only put more demands on more precise information retrieval, querying, and channelling relevant information.****Here, we propose to develop three different core natural language processing methodologies that will make a strong contribution to solving this information management problem. Beside the theoretical results, we develop several tools that actually implement designed solutions, and we also apply these methodologies and tools to specific application areas.****Our approach can be divided into three levels based on the level of language processing: (1) Common N-Gram analysis (CNG), (2) Regular Expression based and finite state processing (RegEx), and (3) unification-based processing and matching of typed feature structures (Unif), with a goal of harmonizing these techniques. The N-gram model, regular expressions, and unification-based grammars are well-understood in NLP. The novelty of our approach is in more specific methodologies developed on top of these models: new n-gram profiling and distance functions, and visualisation; iterative regular expression substitutions, and stochastic unification-based matching and subgraph isomorphism methodology.****