Convex relaxation of problems in data science and efficient solution methods

点击翻译按钮获取中文摘要

Optimization is a fundamental ingredient of machine learning, and furthermore, mathematical tools used to analyze optimization can give some assurance that machine learning methodology gives accurate answers. The proposed research program advances the state of art in optimization in two ways, namely, better modeling of machine learning problems via optimization and better solution algorithms for the resulting optimization problems. A thread running throughout the proposed research is mathematical guarantees for the resulting methods. Among application problems to be tackled are: overlapping community detection, which has applications to social networks, studies of scientific collaboration, and brain science; clustering and nonnegative matrix factorization, two methodologies that discover hidden factors that underlie large data sets; and multiscale computational mechanics, which is the key technology to allow prediction of longevity of large scale structures (buildings, vehicles, and so on) from fundamental principles of physics. Algorithmic development includes better understanding of generalization, that is, the ability of a learning model optimized to classify "training data" to correctly classify future unseen data. Generalization is the key reason why machine learning works at all. The proposed research advances the understanding of generalization by considering "implicit regularization", that is, features of the optimization algorithm such as when to stop iterating that were not specifically designed to improve generalization but nonetheless can be proved to have this effect. The impact of this research is measured in several ways. Students and postdoctoral fellows at Waterloo will receive advanced training in novel and rigorous uses of optimization in machine learning. Within the larger framework of the optimization and machine learning communities, the research will bring newer ideas from optimization to machine learning while at the same time making optimization researchers aware of some challenges in machine learning. The application of machine learning and optimization techniques to multiscale computational mechanics will improve the capability to predict material failures in large-scale structures, particularly for novel materials such as newer fiber composites for which there is not much historical experience. Finally, with regard to society at large, machine learning is proliferating rapidly, and many observers believe that greater understanding and perhaps government regulation is necessary to address concomitant ethical and legal issues. Using mathematical rigor to improve the quality of the computations carried out by machine learning will improve its reliability, which is important to all those affected by this new technology.