EAGER: IIS: Enabling Computationally Efficient Fuzzy Clustering for Distributed Big Data

分布式大数据现在已经广泛使用。由于计算效率和数据隐私的要求，将来自多个分布式站点的所有数据收集到一个地方进行模式分析（称为模糊聚类）具有挑战性。该项目旨在建立一个新的框架，可以对分布式大数据进行有效的模式分析。可以从准确性和计算效率等几个方面深入了解模式分析方法的性能。这项研究的成果将构成分布式大数据环境中模式分析理论和算法的发现和发展的重大进展。这些知识可用于广泛的应用，如医疗保健和运输。该研究项目与一个重要的教育组成部分密切相关。该项目通过课程开发、学生辅导和社区推广招募和教育未来一代的机器学习科学家和工程师。该项目为大数据应用中的模糊聚类奠定了坚实的理论基础。现有的模糊聚类方法缺乏计算效率时，数据分布，非正态和高维，分类和连续变量和缺失值的混合，虽然没有事先假设的统计分布是必需的。在这个项目中，开发了新的方法来提高分布式大数据模糊聚类的效率。该项目的工作包括：（1）为分布式大数据开发计算效率高的模糊聚类;（2）设计分布式大数据智能模糊聚类框架;（3）通过模拟和真实的数据进行性能验证。该项目产生算法、理论模型和实际实施指南，以实现模糊聚类，并为科学研究和工程界开发组件。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Distributed big data are widely available now. Gathering all the data from multiple distributed sites to one place for pattern analysis (called fuzzy clustering) is challenging due to the requirements of computational efficiency and data privacy. This project aims to establish a new framework that can enable efficient pattern analysis for distributed big data. Deep understanding can be gained on the performance of the pattern analysis methods in terms of several metrics, such as accuracy and computational efficiency. The outcomes of this research will constitute a significant advance in the discovery and development of theories and algorithms for pattern analysis in distributed big data environments. This knowledge can be used for wide range of applications, such as healthcare and transportation. The research project is tightly coupled to a vital education component. The project recruits and educates the future generation of machine learning scientists and engineers through curriculum development, student mentoring, and community outreach.This project develops a strong theoretical underpinning for fuzzy clustering for Big Data applications. Existing fuzzy clustering approaches lack computational efficiency when the data are distributed, non-normal and high-dimensional, with a mix of categorical and continuous variables and missing values, although no prior assumptions of statistical distributions are required. In this project, new approaches are developed to augment the efficiency of fuzzy clustering for distributed Big Data. The works in the project include: (1) developing computational efficient fuzzy clustering for distributed Big Data; (2) designing a framework for intelligent fuzzy clustering over distributed Big Data; (3) performance validations through both simulations and real data. This project produces algorithms, theoretical models, and guidelines for practical implementation to enable fuzzy clustering and develop components for the scientific research and engineering communities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.