Theoretical Guarantees of Statistical Methodologies Involving Nonconvex Objectives and the Difference-Of-Convex-Functions Algorithms

这个项目将把涉及非凸优化的统计文献扩展到当代模型。在许多当代机器学习和/或人工智能应用中，利用深度学习和相关神经网络模型。将这些理论扩展到其他当代框架可能会为深度学习等现代技术奠定理论基础。该研究项目具有巨大的潜力，对广大的科学界产生重大影响，他们需要对其庞大的数据进行推理。除了学术出版物和演讲外，这项研究还将为统计和机器学习课程带来新的教学模块。博士学生将得到支持，并接触到渐近理论和计算算法。将开发新的工具箱并在网上提供。软件包的开发，使工程专业的学生（包括本科生）在格鲁吉亚技术和其他大学可以使用他们在他们的课程项目（例如，在工业和系统工程学院本科高级设计项目在格鲁吉亚技术）。PI在过去组织了许多有影响力的研讨会，包括一个关于深度学习的基础，并将继续这样做。具体目标如下。研究工作将把潜在全神经网络模型的统计特性理论推广到其他结构下的神经网络模型，如卷积神经网络，探索推理特性与神经网络结构之间的关系。该项目是为了获得统计估计的理论保证，是基于非凸优化在更一般的设置。PI将探索在基于神经网络的模型中进行类似分析的可能性。统计模型选择可用于偏微分方程的辨识。本课题旨在建立相应的统计理论，揭示相关的实践意义。 将生成一套开放源码软件产品沿着相关文档，以方便我们的工作可复制。将利用现有工具（如GitHub.com或类似工具）传播这些工具。新方法的适用性和需求将在广泛的应用领域中进行探索。具有非凸目标函数的推理技术是许多当代技术中的基本问题，包括基于神经网络的深度学习方法。该项目将有助于这项研究。有明显的社会需求，从大型数据集的推理，这个项目的结果可以有许多应用。该项目将通过探索统计科学的新研究前沿，为统计文献做出贡献。我们的工作是跨学科的，可以在优化和统计社区之间架起桥梁。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will extend the statistical literature that involves nonconvex optimization to contemporary models. In many contemporary machine learning and/or artificial intelligence applications, deep learning and relevant neural network models are utilized. Extending these theories to other contemporary frameworks can potentially lead to a theoretical foundation for modern techniques such as deep learning. The research project has great potential to make a significant impact on the broad scientific community, who have the needs of performing inferences for their enormous data. Besides scholarly publications and presentations, the research will lead to new teaching modules in statistics and machine learning courses. Ph.D. students will be supported and exposed to asymptotic theory and computational algorithms. New toolboxes will be developed and made available online. Packages are developed so that engineering students (including undergraduates) at Georgia Tech and other universities can use them in their course projects (for example, the undergraduate senior design projects at the School of Industrial and Systems Engineering at Georgia Tech). The PI has organized many influential workshops in the past, including one on the foundation of deep learning, and will continue doing so. Specific aims include the following. The research work will extend the theory on the statistical properties of potentially fully neural network models to some other neural network models under different structures, such as the convolutional neural networks, to explore the relation between the inferential property and the neural network architecture. The project is to derive the theoretical guarantees of statistical estimators that are based on nonconvex optimization in more general settings. The PI will explore the possibility to carrying out similar analysis in neural network-based models. Statistical model selection can be utilized in identification of partial differential equations. The project is to establish the corresponding statistical theory and uncover the related practical implication. A set of open-source software products along with related documentation will be generated, to make our work conveniently reproducible. Existing tools (such as GitHub.com or equivalents) will be utilized to disseminate these tools. The applicability and need of the new methods will be explored in a wide spectrum of application domains. Inference techniques with nonconvex objective functions is a fundamental problem in many contemporary techniques, including the neural network based deep learning methodology. This project will contribute to this research. There are evident societal needs for inference from large datasets, and the results of this project can have many applications. The project will contribute to the statistical literature by exploring a new research frontier in statistical sciences. Our work is interdisciplinary and can bridge the communities of optimization and statistics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.