Learning and inference with large image corpora

在我更广泛的计算机视觉和机器学习研究计划中，这项建议针对两个领域：1)大规模数据的深度潜变量模型；2)电子冷冻显微镜三维分子重建算法(Cryo-EM)。我们最近基于两个原则提出了一类新的概率编解码器：1)编解码方一致的对称性；2)观测和潜态之间的高互信息。该公式似乎解决了变分自动编码器的两个问题，即它的不对称损失，与一个近似的编码器，以及它的倾向产生病理结果，称为后部崩溃。初步结果表明，该方法生成的模型交互信息量大，训练稳定。我们的目标是1)完成该模型的理论基础的开发，并使用高维图像和语言数据对其进行实证检验。然后，我们计划将其扩展到支持半监督图像翻译任务，以及大规模语料库上的条件和成分深度表示的学习，并将其应用于少镜头学习。Cryo-EM的目标是在给出电子显微镜的2D图像的原子分辨率下估计3D生物分子结构。我们最近的算法现在被用于一种最先进的软件流水线，称为CryoSPARC。我们对下一代低温电磁算法提出了几个方向：1)我们解决了低温电磁算法中的一个长期存在的问题，即如何以一种基本的方式来衡量估计结构的质量。我们建议开发一种新的原则性公式，作为统计机器学习的一种交叉验证形式；2)我们计划开发用于异质粒子的新算法(而不是现有的方法，假设粒子直到刚性变换都是相同的)。新方法，非均匀细化，将允许信噪比在空间上变化，产生更高的估计结构的分辨率；3)我们计划使用深度学习来去噪估计的3D地图，使用新的技术，而不需要在元学习框架内无噪声的地面真实数据；4)我们计划开发算法来重建灵活的蛋白质，使用简正模式分析，热力学和参数化变形来学习深度条件粒子动力学，产生高动态蛋白质的新算法。影响：无监督学习可能是机器学习的下一个突破，从而避免需要收集大量带注释的训练数据。冷冻-EM在分子生物学和药物发现方面具有颠覆性。这里提出的新方法将保持我们在这一领域的领先地位。最后，对学生进行学习和视觉方面的培训是必不可少的；我的团队之前的HQP都居住在加拿大，包括M Brubaker(Borealis AI)、M Norouzi(Google Brain)、R Urtasun(Uber ATG)和Leonid Sigal(UBC)。

This proposal targets two domains within my broader research program on computer vision and machine learning: 1) deep latent variable models for large-scale data; and 2) algorithms for 3D molecular reconstruction with electron cryo-microscopy (cryo-EM). We recently formulated a new class of probabilistic encoder-decoder based on two principles: 1) symmetry, for which the encoder and decoder are consistent; and 2) high mutual information between observations and latent states. The formulation appears to address 2 problems with variational auto-encoders, ie, its asymmetric loss, with an approximate encoder, and its tendency to produce pathological results, known as posterior collapse. Preliminary results show the approach produces excellent models with high mutually information, and stable training. We aim to 1) to complete the development of the theoretical basis for this model, and test it empirically with high-dimensional image and language data. We then plan to extend it to support semi-supervised image translation tasks, and the learning of conditional and compositional deep representations on large scale corpora, with applications to few-shot learning. The goal of Cryo-EM is to estimate 3D bio-molecular structure at atomic resolutions given 2D images from an electron microscope. Our recent algorithms are now used in a state-of-the-art software pipeline called cryoSPARC. We propose several directions toward the next generation of cryo-EM algorithms: 1) We address a long-standing problem in cryo-EM, namely how to measure the quality of estimated structures in a principaled fashion. We propose to develop a new principled formulation as a form of cross-validation from statistical machine learning; 2) We plan to develop new algorithms for heterogeneous particles (vs current methods that assume particles are identical up to a rigid transform). The new method, non-uniform refinement, will allow signal-to-noise levels to vary spatially, yielding improved resolution of estimated structures; 3) We plan use deep learning to denoise estimated 3D maps, using new techniques that do not reequire noiseless ground truth data, within a meta-learning framework; 4) We plan to develop algorithms for reconstructing flexible proteins using a combination of normal mode analysis, thermodynamics, and parameterized deformations to learn deep conditional particle dynamics, yielding new algorithms for highly dynamic proteins. Impact: Unsupervised learning may be the next breakthrough in machine learning, thereby avoiding to need to collect of vast amounts of annotated training data. Cryo-EM has been disruptive in molecular biology and durg discovery. The new methods proposed here will maintain our leadership in this exicting field. Finally, training students in learning and vision is essential; Previous HQP from my group, all residing in Canada, include M Brubaker (Borealis AI), M Norouzi (Google Brain), R Urtasun (Uber ATG), and Leonid Sigal (UBC).