Convex relaxation of problems in data science and efficient solution methods

优化是机器学习的基本组成部分，此外，用于分析优化的数学工具可以保证机器学习方法给出准确的答案。提出的研究计划以两种方式推进了优化技术的发展，即通过优化对机器学习问题进行更好的建模，以及对结果优化问题进行更好的求解算法。贯穿整个研究的主线是对结果方法的数学保证。需要解决的应用问题包括：重叠社区检测，它可以应用于社会网络、科学协作研究和脑科学；聚类和非负矩阵分解是发现大型数据集背后隐藏因素的两种方法；还有多尺度计算力学，这是一项关键技术，可以从物理学的基本原理来预测大型结构（建筑物、车辆等）的寿命。算法的发展包括更好地理解泛化，即优化学习模型对“训练数据”进行分类的能力，以正确分类未来未见过的数据。泛化是机器学习工作的关键原因。提出的研究通过考虑“隐式正则化”来推进对泛化的理解，即优化算法的特征，如何时停止迭代，不是专门为提高泛化而设计的，但仍然可以证明具有这种效果。这项研究的影响可以从几个方面来衡量。滑铁卢大学的学生和博士后将接受机器学习中新颖而严格的优化应用方面的高级培训。在优化和机器学习社区的大框架内，这项研究将从优化到机器学习带来更新的想法，同时使优化研究人员意识到机器学习中的一些挑战。机器学习和优化技术在多尺度计算力学中的应用将提高预测大尺度结构中材料失效的能力，特别是对于新型材料，如新型纤维复合材料，这类材料没有太多的历史经验。最后，就整个社会而言，机器学习正在迅速扩散，许多观察家认为，为了解决随之而来的道德和法律问题，需要更多的理解和政府监管。使用数学严谨性来提高机器学习执行的计算质量将提高其可靠性，这对所有受这项新技术影响的人都很重要。

Optimization is a fundamental ingredient of machine learning, and furthermore, mathematical tools used to analyze optimization can give some assurance that machine learning methodology gives accurate answers. The proposed research program advances the state of art in optimization in two ways, namely, better modeling of machine learning problems via optimization and better solution algorithms for the resulting optimization problems. A thread running throughout the proposed research is mathematical guarantees for the resulting methods. Among application problems to be tackled are: overlapping community detection, which has applications to social networks, studies of scientific collaboration, and brain science; clustering and nonnegative matrix factorization, two methodologies that discover hidden factors that underlie large data sets; and multiscale computational mechanics, which is the key technology to allow prediction of longevity of large scale structures (buildings, vehicles, and so on) from fundamental principles of physics. Algorithmic development includes better understanding of generalization, that is, the ability of a learning model optimized to classify "training data" to correctly classify future unseen data. Generalization is the key reason why machine learning works at all. The proposed research advances the understanding of generalization by considering "implicit regularization", that is, features of the optimization algorithm such as when to stop iterating that were not specifically designed to improve generalization but nonetheless can be proved to have this effect. The impact of this research is measured in several ways. Students and postdoctoral fellows at Waterloo will receive advanced training in novel and rigorous uses of optimization in machine learning. Within the larger framework of the optimization and machine learning communities, the research will bring newer ideas from optimization to machine learning while at the same time making optimization researchers aware of some challenges in machine learning. The application of machine learning and optimization techniques to multiscale computational mechanics will improve the capability to predict material failures in large-scale structures, particularly for novel materials such as newer fiber composites for which there is not much historical experience. Finally, with regard to society at large, machine learning is proliferating rapidly, and many observers believe that greater understanding and perhaps government regulation is necessary to address concomitant ethical and legal issues. Using mathematical rigor to improve the quality of the computations carried out by machine learning will improve its reliability, which is important to all those affected by this new technology.