EAGER: IIS: Enabling Computationally Efficient Fuzzy Clustering for Distributed Big Data

如今，分布式大数据已经广泛应用。由于计算效率和数据隐私的要求，将来自多个分布式站点的所有数据收集到一个地方进行模式分析（称为模糊聚类）是具有挑战性的。该项目旨在建立一个新的框架，能够对分布式大数据进行高效的模式分析。深入了解模式分析方法的几个指标，如准确性和计算效率方面的性能。本研究的结果将构成分布式大数据环境中模式分析理论和算法的发现和发展的重大进步。这些知识可用于广泛的应用，例如医疗保健和运输。这个研究项目与一个重要的教育组成部分紧密相连。该项目通过课程开发、学生指导和社区外展来招募和教育下一代机器学习科学家和工程师。本项目为大数据应用中的模糊聚类提供了强有力的理论基础。当数据是分布的、非正态的、高维的、混合了分类变量和连续变量以及缺失值时，现有的模糊聚类方法缺乏计算效率，尽管不需要事先假设统计分布。在这个项目中，开发了新的方法来提高分布式大数据模糊聚类的效率。项目工作包括：(1)为分布式大数据开发计算效率高的模糊聚类；(2)设计分布式大数据智能模糊聚类框架；(3)通过仿真和实际数据进行性能验证。该项目产生算法、理论模型和指导方针，以实现模糊聚类，并为科学研究和工程界开发组件。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Distributed big data are widely available now. Gathering all the data from multiple distributed sites to one place for pattern analysis (called fuzzy clustering) is challenging due to the requirements of computational efficiency and data privacy. This project aims to establish a new framework that can enable efficient pattern analysis for distributed big data. Deep understanding can be gained on the performance of the pattern analysis methods in terms of several metrics, such as accuracy and computational efficiency. The outcomes of this research will constitute a significant advance in the discovery and development of theories and algorithms for pattern analysis in distributed big data environments. This knowledge can be used for wide range of applications, such as healthcare and transportation. The research project is tightly coupled to a vital education component. The project recruits and educates the future generation of machine learning scientists and engineers through curriculum development, student mentoring, and community outreach.This project develops a strong theoretical underpinning for fuzzy clustering for Big Data applications. Existing fuzzy clustering approaches lack computational efficiency when the data are distributed, non-normal and high-dimensional, with a mix of categorical and continuous variables and missing values, although no prior assumptions of statistical distributions are required. In this project, new approaches are developed to augment the efficiency of fuzzy clustering for distributed Big Data. The works in the project include: (1) developing computational efficient fuzzy clustering for distributed Big Data; (2) designing a framework for intelligent fuzzy clustering over distributed Big Data; (3) performance validations through both simulations and real data. This project produces algorithms, theoretical models, and guidelines for practical implementation to enable fuzzy clustering and develop components for the scientific research and engineering communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.