On the Feasibility of Distributed Statistical Learning for Big Data

收集前所未有的规模和复杂性的数据现在在许多科学领域都是可行的。大数据只有在被有效利用以获取有用信息时才具有战略价值。开发有效的工具来管理和处理大数据一直是统计学和相关学科的热点。 由于其庞大的体积，大数据很少可以在一台机器上存储和处理。因此，为了计算方便，通常使用分治方案。在这种策略中，完整的数据集被分成几个可管理的部分并在其中进行处理;然后从分段的子输出中聚合最终输出。尽管这种分配框架在实践中很受欢迎，但它缺乏坚实的理论基础。其性能在不同的场景中可能会有所不同。因此，有必要对该方法进行改进，并提供相关的理论支持。在这个提案中，我计划系统地研究几个主要统计学习任务的分配方法。具体而言，我有以下三个目标。 1)我将详细说明条件，并建立回归、分类和排序目的的简单分配方法的一致性。这一目标将提供一个基本的理论认识的分配框架，以便更好地指导其应用。2)基于分布式优化，我将设计新的学习过程，改进简单的分配方法。这些新方法将增强各个机器之间的通信。因此，它们有很大的潜力使整个程序更加有效和可靠。3)我将把开发的分布式方法应用于现实世界的数据集。这一目标将为新方法提供一个软件包，并解决实践中提出的执行问题。一个潜在的应用是改善加拿大住院病人所需的医院资源的预测系统。本申请将基于加拿大健康信息研究所提供的2015-2020年出院摘要数据库。 通过上述研究目标，该计划将为分布式统计学习奠定理论基础并制定有效的实施程序。这些结果和产品将成为处理大数据的关键技术之一。拟议的研究目标将被视为和调查的统计，机器学习，优化和近似理论的角度联合。 这一切都使得该方案在新兴的大数据领域进行了新颖而有前途的探索。该计划的多元化主题非常适合培养博士和硕士水平的高素质人才。该计划培养的科学家迫切需要在工业，研究机构和政府机构分析大数据。

Collecting data with unprecedented sizes and complexities is now feasible in many scientific fields. Big data have strategic value only when they are effectively utilized to obtain useful information. Developing efficient tools to manage and process big data has been a recent hotspot in statistics and related disciplines. Due to their huge volume, big data can be rarely stored and processed on a single machine. Therefore, a divide-and-conquer scheme is often used for computational convenience. In such a strategy, a full dataset is split into and processed in several manageable segments; the final output is then aggregated from the segmental sub-outputs. Despite its practical popularity, this distributive framework lacks a solid theoretical foundation. Its performance can vary in different scenarios. Thus, it is necessary to refine the method and provide the associated theoretical support. In this proposal, I plan to systematically investigate the distributive method for a few major statistical learning tasks. Specifically, I have the following three objectives. 1) I will specify the conditions and establish the consistency of simple distributive methods for regression, classification, and ranking purposes. This objective will provide a basic theoretical understanding of the distributive framework, so that better guidance can be provided for its application. 2) Based on distributed optimization, I will design new learning procedures that improve the simple distributive method. The new methods will enhance the communication between individual machines. Thus, they have great potential to make the overall procedure more efficient and reliable. 3) I will apply the developed distributive methods to real-world datasets. This objective will provide a software package for the new methods and address the implementation issues raised in practice. One potential application is improving the prediction system for hospital resources needed by Canadian inpatients. This application will be based on 2015-2020 Discharge Abstract Database available from Canadian Institute for Health Information. With the above research objectives, this program will build a theoretical foundation and develop efficient implementation procedures for distributive statistical learning. These results and products will be among the key techniques for processing big data. The proposed research objectives will be viewed and investigated jointly from the perspectives of statistics, machine learning, optimization, and approximation theory. All these make this program a novel and promising exploration in the emerging field of big data. The diversified topics in this program are well-suited for training highly qualified personals at both doctoral and master's levels. The scientists trained by this program are urgently needed for analyzing big data in industry, research institutes, and government agencies.