Theoretical Guarantees of Statistical Methodologies Involving Nonconvex Objectives and the Difference-Of-Convex-Functions Algorithms

本项目将扩展涉及非凸优化的统计文献到当代模型。在许多当代机器学习和/或人工智能应用中，使用了深度学习和相关的神经网络模型。将这些理论扩展到其他当代框架中，可能会为深度学习等现代技术提供理论基础。该研究项目具有巨大的潜力，可以对广泛的科学界产生重大影响，因为科学界需要对其庞大的数据进行推断。除了学术出版物和演讲，这项研究还将在统计学和机器学习课程中引入新的教学模块。博士生将被支持并接触到渐近理论和计算算法。将开发新的工具箱并在网上提供。开发的软件包是为了让佐治亚理工学院和其他大学的工程专业学生（包括本科生）可以在他们的课程项目中使用它们（例如，佐治亚理工学院工业与系统工程学院的本科高级设计项目）。PI过去组织了许多有影响力的研讨会，包括深度学习基础研讨会，并将继续这样做。具体目标包括以下内容。本研究将把潜在全神经网络模型的统计性质理论扩展到其他不同结构的神经网络模型，如卷积神经网络，以探索推理性质与神经网络结构之间的关系。该项目是在更一般的情况下推导基于非凸优化的统计估计器的理论保证。PI将探索在基于神经网络的模型中进行类似分析的可能性。统计模型选择可以用于偏微分方程的辨识。本课题旨在建立相应的统计理论，揭示相关的现实意义。将生成一组开源软件产品以及相关文档，使我们的工作方便地再现。现有的工具（如GitHub.com或同等工具）将被用来传播这些工具。新方法的适用性和需求将在广泛的应用领域进行探索。基于非凸目标函数的推理技术是许多当代技术中的一个基本问题，包括基于神经网络的深度学习方法。这个项目将有助于这项研究。社会对大型数据集的推断有明显的需求，这个项目的结果可以有很多应用。该项目将通过探索统计科学的新研究前沿，为统计文献做出贡献。我们的工作是跨学科的，可以桥梁社区的优化和统计。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will extend the statistical literature that involves nonconvex optimization to contemporary models. In many contemporary machine learning and/or artificial intelligence applications, deep learning and relevant neural network models are utilized. Extending these theories to other contemporary frameworks can potentially lead to a theoretical foundation for modern techniques such as deep learning. The research project has great potential to make a significant impact on the broad scientific community, who have the needs of performing inferences for their enormous data. Besides scholarly publications and presentations, the research will lead to new teaching modules in statistics and machine learning courses. Ph.D. students will be supported and exposed to asymptotic theory and computational algorithms. New toolboxes will be developed and made available online. Packages are developed so that engineering students (including undergraduates) at Georgia Tech and other universities can use them in their course projects (for example, the undergraduate senior design projects at the School of Industrial and Systems Engineering at Georgia Tech). The PI has organized many influential workshops in the past, including one on the foundation of deep learning, and will continue doing so. Specific aims include the following. The research work will extend the theory on the statistical properties of potentially fully neural network models to some other neural network models under different structures, such as the convolutional neural networks, to explore the relation between the inferential property and the neural network architecture. The project is to derive the theoretical guarantees of statistical estimators that are based on nonconvex optimization in more general settings. The PI will explore the possibility to carrying out similar analysis in neural network-based models. Statistical model selection can be utilized in identification of partial differential equations. The project is to establish the corresponding statistical theory and uncover the related practical implication. A set of open-source software products along with related documentation will be generated, to make our work conveniently reproducible. Existing tools (such as GitHub.com or equivalents) will be utilized to disseminate these tools. The applicability and need of the new methods will be explored in a wide spectrum of application domains. Inference techniques with nonconvex objective functions is a fundamental problem in many contemporary techniques, including the neural network based deep learning methodology. This project will contribute to this research. There are evident societal needs for inference from large datasets, and the results of this project can have many applications. The project will contribute to the statistical literature by exploring a new research frontier in statistical sciences. Our work is interdisciplinary and can bridge the communities of optimization and statistics.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.