String-based and Unification-based Methodology for Text mining and Processing

在当今时代，信息技术对经济和社会的影响在所谓的社交媒体和更广泛意义上的互联网领域可能是最直接和最重要的。为Web提供可访问内容的人的比例已经从一小部分变成了很大一部分，很可能是大多数。已经确定的是，这种“大数据”的洪流一方面可以增加我们的知识和效率，从而带来更大的社会效益，但另一方面，我们经常迷失，淹没在无关的信息中，而错过了相关的信息。移动技术无处不在，但受屏幕尺寸、用户输入速率和处理能力的限制，移动技术的普及只会对更精确的信息检索、查询和引导相关信息提出更多要求。****在这里，我们建议开发三种不同的核心自然语言处理方法，这些方法将为解决这一信息管理问题做出强有力的贡献。除了理论结果之外，我们还开发了一些实际实现设计解决方案的工具，并且我们还将这些方法和工具应用于特定的应用领域。****我们的方法可以根据语言处理的水平分为三个层次：(1)通用N-Gram分析（CNG），(2)基于正则表达式和有限状态处理（RegEx），以及(3)基于统一的处理和匹配类型特征结构（Unif），目标是协调这些技术。N-gram模型、正则表达式和基于统一的语法在NLP中得到了很好的理解。我们方法的新颖之处在于在这些模型之上开发的更具体的方法：新的n图分析和距离函数，以及可视化；迭代正则表达式替换，以及基于随机统一的匹配和子图同构方法。****

The economic and social impact of information technology in the current time is probably most direct and significant in the area of the so-called social media and Internet in a wider sense. The ratio of people contributing accessible content to the Web has gone from a minor fraction to a large portion, likely a majority. It has been established that this flood of "big data" on one side can increase our knowledge and efficiency, leading to a greater social benefit, but on another side we are frequently lost and drowning in irrelevant information while missing relevant information. The proliferation of mobile technology, which is available anywhere but also limited in screen size, user input rate and processing power, has only put more demands on more precise information retrieval, querying, and channelling relevant information.****Here, we propose to develop three different core natural language processing methodologies that will make a strong contribution to solving this information management problem. Beside the theoretical results, we develop several tools that actually implement designed solutions, and we also apply these methodologies and tools to specific application areas.****Our approach can be divided into three levels based on the level of language processing: (1) Common N-Gram analysis (CNG), (2) Regular Expression based and finite state processing (RegEx), and (3) unification-based processing and matching of typed feature structures (Unif), with a goal of harmonizing these techniques. The N-gram model, regular expressions, and unification-based grammars are well-understood in NLP. The novelty of our approach is in more specific methodologies developed on top of these models: new n-gram profiling and distance functions, and visualisation; iterative regular expression substitutions, and stochastic unification-based matching and subgraph isomorphism methodology.****