Algorithm and systems engineering for high-performance visual text analytics on big data

大数据是一个流行的术语，用于描述数据的指数增长和可用性以及它所带来的技术机遇和挑战。在许多不同的应用中，大数据挑战是如何从极其庞大的数据集合中推断出重要的趋势和因果关系。例如，分析航空事件和事故报告，将组件故障与导致故障发生的操作条件联系起来，分析产品评论中提供的文本反馈，以改进产品，或分析Facebook和Twitter帖子中的热门话题。这些示例的共同主题是，至少大部分信息是以非结构化文本形式存储的。最近的研究已经成功地开发了机器学习技术，通过按主题对文档进行分组，从文档中提取最重要的关键字以便于用户查看，等等来支持对文本文档集合的分析。这些技术中的大多数具有高计算成本，这限制了它们对非常大的文本集合的适用性。通过开发、实施、性能评估和调整用于这些机器学习技术基础的基本分析任务的改进算法，以及通过使用并行计算和云技术，可以大幅提高性能。这将使这些技术的应用显着更大的文本集合，是本研究的重点。

Big Data is a popular term used to describe the exponential growth and availability of data and the technical opportunities and challenges it presents. In many diverse applications, the Big Data Challenge is how to infer important trends and causal relationships from extremely large data collections. Examples include the analysis of aviation incident and accident reports to link component failures to operational conditions that cause them to occur, analysis of textual feedback provided in product reviews with the goal to improve the products or the analysis of trending topics in Facebook and Twitter posts. The common theme of these examples is that at least a large part of the information is stored in unstructured textual form. Recent research has been successful in developing machine learning techniques that support the analysis of collections of text documents by grouping documents by topic, extracting the most important keywords from documents for easy review by the user, and many more. Most of these techniques have a high computation cost, which limits their applicability to very large text collections. Substantial performance improvements are possible through the development, implementation, performance evaluation, and tuning of improved algorithms for the basic analysis tasks underlying these machine learning techniques and through the use of parallel computing and cloud technologies. This will enable the application of these techniques to significantly larger text collections and is the focus of this research.