Convex relaxation of problems in data science and efficient solution methods

优化是机器学习的一个基本组成部分，此外，用于分析优化的数学工具可以在一定程度上保证机器学习方法给出准确的答案。提出的研究方案从两个方面推动了优化领域的发展，即通过优化对机器学习问题进行更好的建模，以及对所产生的优化问题进行更好的求解算法。贯穿整个拟议研究的一个线索是对结果方法的数学保证。需要解决的应用问题包括：重叠社区检测，它应用于社会网络、科学合作研究和脑科学；聚类和非负矩阵分解，这两种方法可以发现大型数据集背后的隐藏因素；以及多尺度计算力学，这是允许从物理学基本原理预测大型结构(建筑物、车辆等)寿命的关键技术。算法的发展包括更好地理解泛化，即优化学习模型以分类“训练数据”的能力，以正确地分类未来看不见的数据。泛化是机器学习奏效的关键原因。这项研究通过考虑“隐含正则化”来提高对泛化的理解，即优化算法的特征，例如何时停止迭代，这些特征并不是专门为改进泛化而设计的，但仍然可以被证明具有这种效果。这项研究的影响可以通过几种方式来衡量。滑铁卢大学的学生和博士后研究员将接受关于机器学习中优化的新颖和严格使用的高级培训。在优化和机器学习社区的更大框架内，这项研究将把从优化到机器学习的新想法带到机器学习，同时让优化研究人员意识到机器学习中的一些挑战。将机器学习和优化技术应用到多尺度计算力学中，将提高预测大型结构中材料失效的能力，特别是对于历史经验不多的新型材料，如较新的纤维复合材料。最后，就整个社会而言，机器学习正在迅速扩散，许多观察家认为，为了解决随之而来的伦理和法律问题，需要更多的理解，或许还有政府的监管。使用数学严谨性来提高机器学习进行的计算质量将提高其可靠性，这对所有受这项新技术影响的人来说都是重要的。

Optimization is a fundamental ingredient of machine learning, and furthermore, mathematical tools used to analyze optimization can give some assurance that machine learning methodology gives accurate answers. The proposed research program advances the state of art in optimization in two ways, namely, better modeling of machine learning problems via optimization and better solution algorithms for the resulting optimization problems. A thread running throughout the proposed research is mathematical guarantees for the resulting methods. Among application problems to be tackled are: overlapping community detection, which has applications to social networks, studies of scientific collaboration, and brain science; clustering and nonnegative matrix factorization, two methodologies that discover hidden factors that underlie large data sets; and multiscale computational mechanics, which is the key technology to allow prediction of longevity of large scale structures (buildings, vehicles, and so on) from fundamental principles of physics. Algorithmic development includes better understanding of generalization, that is, the ability of a learning model optimized to classify "training data" to correctly classify future unseen data. Generalization is the key reason why machine learning works at all. The proposed research advances the understanding of generalization by considering "implicit regularization", that is, features of the optimization algorithm such as when to stop iterating that were not specifically designed to improve generalization but nonetheless can be proved to have this effect. The impact of this research is measured in several ways. Students and postdoctoral fellows at Waterloo will receive advanced training in novel and rigorous uses of optimization in machine learning. Within the larger framework of the optimization and machine learning communities, the research will bring newer ideas from optimization to machine learning while at the same time making optimization researchers aware of some challenges in machine learning. The application of machine learning and optimization techniques to multiscale computational mechanics will improve the capability to predict material failures in large-scale structures, particularly for novel materials such as newer fiber composites for which there is not much historical experience. Finally, with regard to society at large, machine learning is proliferating rapidly, and many observers believe that greater understanding and perhaps government regulation is necessary to address concomitant ethical and legal issues. Using mathematical rigor to improve the quality of the computations carried out by machine learning will improve its reliability, which is important to all those affected by this new technology.