Random Hessians and Jacobians: theory and applications

点击翻译按钮获取中文摘要

Properties of complicated 'landscapes', i.e. randomfunctions defined on very high dimensional spaces, have recently attracted considerable attention, e.g. in theory of Deep Machine Learning and Optimization. In particular, one may be interested in number of 'valleys' (i.e. local minima) at a given 'height', 'ridges' or barriers separating them, and more generally 'critical points' (saddles and maxima). An important role in characterising geometry of the landscapes, especially close to the critical points, is played by the matrix of second derivatives known as the Hessian. It determines e.g. the gradient descent dynamics within these landscapes, which has many practical applications for search algorithms. Depending on the context, the landscape can correspond to the energy of a physical system, to the loss function of a machine-learning algorithm, to the cost function of an optimization problem, or tothe fitness function of a biological system. In the analysis of critical points the (modulus of) the characteristic polynomial of the Hessian appears naturally. Similarly, to characterize equilibria in complicated dynamical systems (e.g. communities of many interacting species) requires investigating properties of more general, asymmetric, Jacobian matrices, for which Hessians are only a special case. Jacobians are deeply related to questions of stability of systems under small perturbations, and as such are very fundamental. Note that in contrast to Hessians whose spectra are real and eigenvectors form an orthogonal set, the Jacobians have in general complex eigenvalues and bi-orthogonal set of left and right eigenvectors. The studies of the associated 'eigenvector non-orthogonality' in random setting turn out to be relevant both for complex systems stability as well as to chaotic wave scattering and random lasing. The present research proposal is mainly centred around analysis of various properties of random matrices and operators, mostly arising via Hessians of random landscapes, or random Jacobians of various origin.