Collaborative Knowledge Discovery in Digital Government Data Using Distributed Higher-Order Text Mining

分布式源中文本数据的迅速增长，以及创建和维护中央存储库的障碍，促使人们需要有效的分布式信息提取和挖掘技术。一个人的不同类型的记录可能存在于不同的数据库中，这是一种数据碎片。 然而，即使有了标准，自动集成模式的能力也是一个开放的研究问题。 一个相关的问题是，目前的关联规则挖掘（ARM）算法挖掘分布式数据的能力挖掘数据（无论是垂直或水平碎片），只有当所有数据库的全局模式是已知的。 在从分布式文本数据中提取信息的情况下，没有预先存在的全局模式可用。这是由于提取的实体在文档之间不同-新输入文本可能包含以前未见过的实体。其结果是，一个固定的全球模式不能被假定和现有的算法不能employe.This努力描述了一个分布式的高阶文本挖掘框架，既不需要知识的全球模式，也不模式集成为前驱挖掘规则。 该框架被称为D-HOTM，提取实体并基于由公共密钥链接的记录中的实体之间的高阶关联来发现规则。 实体提取是基于使用半监督主动学习算法先前开发的信息提取规则。 学习的规则被应用于从描述例如犯罪作案手法的文本数据中自动提取实体。提取的实体存储在本地关系数据库中，这些数据库使用D-HOTM分布式关联规则挖掘算法进行挖掘。这项工作的更广泛影响在于与当地执法部门和医疗保健提供商合作，部署现场测试床，通过挖掘报告和识别医生最佳实践来解决问题。为学生提供大学预科实习机会，并为研究生提供支持。

ABSTRACTNSF-0534276Pottenger, WilliamThe burgeoning amount of textual data in distributed sources combined with the obstacles involved in creating and maintaining central repositories motivates the need for effective distributed information extraction and mining techniques. Different kinds of records on a given individual may exist in different databases - a type of data fragmentation. Even with standards, however, the ability to integrate schemas automatically is an open research issue. A related issue is the fact that current Association Rule Mining (ARM) algorithms for mining distributed data are capable of mining data (whether vertically or horizontally fragmented) only when the global schema across all databases is known. In the case of information extracted from distributed textual data, no preexisting global schema is available. This is due to the fact that the entities extracted vary between documents - new input text can contain previously unseen entities. As a result, a fixed global schema cannot be assumed and existing algorithms cannot be employed.This effort describes a distributed higher-order text mining framework that requires neither the knowledge of the global schema nor schema integration as a precursor to mining rules. The framework, termed D-HOTM, extracts entities and discovers rules based on higher-order associations between entities in records linked by a common key. The entity extraction is based on information extraction rules learned using a semi-supervised active learning algorithm previously developed. The rules learned are applied to automatically extract entities from textual data that describe, for example, criminal modus operandi. The entities extracted are stored in local relational databases, which are mined using the D-HOTM distributed association rule mining algorithm.The broader impacts of thework lie in the collaboration with local law enforcement and healthcare providers for deploying live test beds that enable problem solving by mining reports and identificaiton of physician best practices. Pre-college internships are provided for students as well as support for graduate students.