Learning and inference with large image corpora

点击翻译按钮获取中文摘要

This proposal targets two domains within my broader research program on computer vision and machine learning: 1) deep latent variable models for large-scale data; and 2) algorithms for 3D molecular reconstruction with electron cryo-microscopy (cryo-EM). We recently formulated a new class of probabilistic encoder-decoder based on two principles: 1) symmetry, for which the encoder and decoder are consistent; and 2) high mutual information between observations and latent states. The formulation appears to address 2 problems with variational auto-encoders, ie, its asymmetric loss, with an approximate encoder, and its tendency to produce pathological results, known as posterior collapse. Preliminary results show the approach produces excellent models with high mutually information, and stable training. We aim to 1) to complete the development of the theoretical basis for this model, and test it empirically with high-dimensional image and language data. We then plan to extend it to support semi-supervised image translation tasks, and the learning of conditional and compositional deep representations on large scale corpora, with applications to few-shot learning. The goal of Cryo-EM is to estimate 3D bio-molecular structure at atomic resolutions given 2D images from an electron microscope. Our recent algorithms are now used in a state-of-the-art software pipeline called cryoSPARC. We propose several directions toward the next generation of cryo-EM algorithms: 1) We address a long-standing problem in cryo-EM, namely how to measure the quality of estimated structures in a principaled fashion. We propose to develop a new principled formulation as a form of cross-validation from statistical machine learning; 2) We plan to develop new algorithms for heterogeneous particles (vs current methods that assume particles are identical up to a rigid transform). The new method, non-uniform refinement, will allow signal-to-noise levels to vary spatially, yielding improved resolution of estimated structures; 3) We plan use deep learning to denoise estimated 3D maps, using new techniques that do not reequire noiseless ground truth data, within a meta-learning framework; 4) We plan to develop algorithms for reconstructing flexible proteins using a combination of normal mode analysis, thermodynamics, and parameterized deformations to learn deep conditional particle dynamics, yielding new algorithms for highly dynamic proteins. Impact: Unsupervised learning may be the next breakthrough in machine learning, thereby avoiding to need to collect of vast amounts of annotated training data. Cryo-EM has been disruptive in molecular biology and durg discovery. The new methods proposed here will maintain our leadership in this exicting field. Finally, training students in learning and vision is essential; Previous HQP from my group, all residing in Canada, include M Brubaker (Borealis AI), M Norouzi (Google Brain), R Urtasun (Uber ATG), and Leonid Sigal (UBC).