Collaborative Research: New Statistical Learning and Scalable Computing for Large Unstructured Data

这一建议侧重于一些基本问题，涉及非结构化数据，从文本密集的文件，其中的基础数据表现出独特的特点，如大容量，大品种和大的变化速度。信息提取过程的自动化在信息时代至关重要，在在线调查、威胁检测和预防中具有很高的实用性。研究和教育的综合计划将在许多领域产生重大影响，如机器学习和数据挖掘，自然语言处理，民意调查，商业预测和服务，健康研究以及社会和政治科学等。这将促进跨学科研究和与来自不同领域的科学家的合作。拟议的项目需要广泛的算法和软件开发的目标应用程序。特别是，将在分布式计算平台（如OpenMP、MPI和Hadoop）上通过mapReduce开发先进的计算工具，并将沿着技术转让传播软件文档。非结构化数据带来了巨大的挑战，因为文本文档需要嵌入并与数字输入集成，以进行统计建模，这需要过参数化建模来实现对高维数据的准确预测和无偏推断。建议的研究旨在开发新的统计方法和工具，情感分析和文本摘要利用词的关系，通过图形和个性化的预测推荐系统。 它借用所有可用信息的信息进行文档摘要，包括标记和未标记的文档，从而提高标记的准确性。 这将加强信息的储存、分类和处理以及过滤。此外，该项目还开发了一种新的方法，利用所有用户之间的异质性变化进行准确的个性化预测，这会影响日常生活中的个性化，例如服务，推荐和广告。更重要的是，所提出的统计方法和可扩展的计算算法对于其他类型的非结构化数据将是有价值和有用的。最后，许多先进的优化技术和计算程序也将适用于其他类型的“大”数据问题。

This proposal focuses on some fundamental issues concerning unstructured data that arise from text-heavy documents, where the underlying data exhibit unique characteristics such as large volume, large variety and large velocity of change. Automating the process of information extraction is extremely critical in the information age, and has high-utility in online surveys, and threat detection and prevention. The integrated program of research and education will have significant impacts in many fields such as machine learning and data mining, natural language processing, opinion survey, business forecasting and service, health research, and social and political science, among others. This will stimulate interdisciplinary research and collaboration with scientists from disparate fields. The proposed project requires extensive algorithm and software development for target applications. In particular, advanced computational tools will be developed through mapReduce over distributed computational platforms such as OpenMP, MPI and hadoop, and documentation of the software will be disseminated along with the technology transfer.Unstructured data impose great challenges in that text documents need to be embedded and integrated with numerical input for statistical modeling, which requires overparameterized modeling to achieve accurate prediction and unbiased inference for high-dimensional data. The proposed research aims to develop new statistical methods and tools for sentiment analysis and text summarization utilizing word relations through graphs and personalized prediction for recommender systems. It borrows information across all available information for document summarization, including tagged and untagged documents, leading to higher accuracy of tagging. This will enhance information storage, sorting and processing as well as filtering. Moreover, the project develops a novel approach for accurate personalized prediction utilizing the heterogeneity variation among all users, which impacts everyday life in terms of personalization, such as in service, recommendation and advertising. More importantly, the proposed statistical methodology and scalable computational algorithms will be valuable and useful for other types of unstructured data. Finally, many of the advanced optimization techniques and computing procedures to be developed will also be applicable to other types of ``BIG" data problems.