CSR: Small: Data Parallel Frameworks for Large-scale Machine Learning through Sync-on-the-Fly

传感、存储和网络技术的进步导致了大容量、高维数据的收集。理解这些数据对于公司和组织做出更好的商业决策，为我们的日常生活带来便利，甚至通过生物信息和药物发现实现更好的健康至关重要。机器学习的最新进展导致了一系列通常需要迭代改进过程的数据分析技术。然而，涉及的大量数据和可能需要的大量迭代使得及时执行数据分析具有挑战性。该项目旨在设计和实现一个名为同步-动态的数据并行编程框架。该框架使云计算环境中的机器学习计算能够在计算执行期间建立同步屏障。由于同步障碍是为构建一致的模型参数而建立的，因此不需要对它们进行预定义。在计算过程中，可以根据计算的进度建立障碍。这种数据并行编程模型保留了机器学习算法的语义。目标是为一系列知名的机器学习算法建立理论基础和创建高效的分布式框架，并为这些计算建立编程模型。该项目开发的技术将在道路交通预测、生物信息发现、在线营销和计算机法医分析等方面具有直接的重要应用。该项目将为用户带来快速、准确、高性价比的海量数据处理。该项目还将培训分布式框架设计、机器学习算法和大数据分析方面的新毕业生工程师。所有这些技能在美国工业中都有广泛的需求。为分布式框架和分布式机器学习算法生成的数据和软件代码将在研究网站http://rio.ecs.umass.edu/html/research/index.html上公开。该网站将在项目结束后至少保留三年。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The advances in sensing, storage, and networking technologies have led to the collections of high-volume, high-dimensional data. Making sense of these data is critical for companies and organizations to make better business decisions, to bring convenience to our daily life, and even enable better health through biological information and drug discovery. Recent advances in machine learning have led to a flurry of data analytic techniques that typically require an iterative refinement process. However, the massive amount of data involved and potentially numerous iterations required make performing data analytics in a timely manner challenging.This project aims to design and implement a data parallel programming framework called Sync-on-the-fly. The framework enables machine learning computations within cloud computing environments to establish synchronization barriers during the execution of the computations. Since synchronization barriers are established for building consistent model parameters, they do not need to be predefined. The barriers can be established during the computation based on the progress of the computation. This data parallel programming model preserves the semantics of machine learning algorithms. The goals are to build theoretical foundations and create efficient distributed frameworks for a series of well-known machine learning algorithms, and establish the programming models for these computations.The technologies developed from this project will have immediate important applications on road traffic prediction, biological information discovery, online marketing, and computer forensic analysis. This project will bring fast, accurate, and cost-effective processing of massive data to users. This project will also train new graduate engineers in distributed framework design, machine learning algorithms, and big data analytics. All of these skillsets are in broad demand in US industry.The data and software codes produced for the distributed framework and distributed machine learning algorithms will be made publicly available at the research website http://rio.ecs.umass.edu/html/research/index.html. The website will be retained for at least three years after the conclusion of the project.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.