Learning and inference with large image corpora

该提案针对我在计算机视觉和机器学习方面更广泛的研究计划中的两个领域：1）大规模数据的深层潜变量模型; 2）电子冷冻显微镜（cryo-EM）的3D分子重建算法。 我们最近基于两个原则制定了一类新的概率编码器-解码器：1）对称性，编码器和解码器是一致的; 2）观察和潜在状态之间的高互信息。该公式似乎解决了2个问题与变分自动编码器，即，其不对称损失，与一个近似的编码器，其倾向于产生病理结果，被称为后崩溃。初步结果表明，该方法产生了良好的模型，具有高互信息，稳定的训练。 我们的目标是：1）完成该模型的理论基础的开发，并使用高维图像和语言数据对其进行实证测试。然后，我们计划扩展它以支持半监督图像翻译任务，以及在大规模语料库上学习条件和组合深度表示，并应用于少量学习。 Cryo-EM的目标是在电子显微镜给出的2D图像下以原子分辨率估计3D生物分子结构。我们最近的算法现在被用于一个名为cryogenic的最先进的软件管道中。我们提出了下一代cryo-EM算法的几个方向：1）我们解决了cryo-EM中一个长期存在的问题，即如何以一种原则性的方式测量估计结构的质量。 我们建议开发一种新的原则性公式，作为统计机器学习交叉验证的一种形式; 2）我们计划为异质粒子开发新的算法（与目前假设粒子在刚性变换之前是相同的方法相比）。 新方法，非均匀细化，将允许信噪比水平在空间上变化，从而提高估计结构的分辨率; 3）我们计划使用深度学习来对估计的3D地图进行降噪，使用不需要无噪声地面真实数据的新技术，在元学习框架内; 4）我们计划开发使用正常模式分析，热力学和参数化变形的组合来重建柔性蛋白质的算法，以学习深度条件粒子动力学，从而产生了针对高动态蛋白质的新算法。 影响：无监督学习可能是机器学习的下一个突破，从而避免需要收集大量带注释的训练数据。 冷冻电镜在分子生物学和杜尔格发现方面具有颠覆性。 这里提出的新方法将保持我们在这一领域的领导地位。最后，对学生进行学习和视觉方面的培训是必不可少的;我所在团队的前HQP都居住在加拿大，包括M Brubaker（Borealis AI），M Norouzi（Google Brain），R Urtasun（Uber ATG）和Leonid Sigal（UBC）。

This proposal targets two domains within my broader research program on computer vision and machine learning: 1) deep latent variable models for large-scale data; and 2) algorithms for 3D molecular reconstruction with electron cryo-microscopy (cryo-EM). We recently formulated a new class of probabilistic encoder-decoder based on two principles: 1) symmetry, for which the encoder and decoder are consistent; and 2) high mutual information between observations and latent states. The formulation appears to address 2 problems with variational auto-encoders, ie, its asymmetric loss, with an approximate encoder, and its tendency to produce pathological results, known as posterior collapse. Preliminary results show the approach produces excellent models with high mutually information, and stable training. We aim to 1) to complete the development of the theoretical basis for this model, and test it empirically with high-dimensional image and language data. We then plan to extend it to support semi-supervised image translation tasks, and the learning of conditional and compositional deep representations on large scale corpora, with applications to few-shot learning. The goal of Cryo-EM is to estimate 3D bio-molecular structure at atomic resolutions given 2D images from an electron microscope. Our recent algorithms are now used in a state-of-the-art software pipeline called cryoSPARC. We propose several directions toward the next generation of cryo-EM algorithms: 1) We address a long-standing problem in cryo-EM, namely how to measure the quality of estimated structures in a principaled fashion. We propose to develop a new principled formulation as a form of cross-validation from statistical machine learning; 2) We plan to develop new algorithms for heterogeneous particles (vs current methods that assume particles are identical up to a rigid transform). The new method, non-uniform refinement, will allow signal-to-noise levels to vary spatially, yielding improved resolution of estimated structures; 3) We plan use deep learning to denoise estimated 3D maps, using new techniques that do not reequire noiseless ground truth data, within a meta-learning framework; 4) We plan to develop algorithms for reconstructing flexible proteins using a combination of normal mode analysis, thermodynamics, and parameterized deformations to learn deep conditional particle dynamics, yielding new algorithms for highly dynamic proteins. Impact: Unsupervised learning may be the next breakthrough in machine learning, thereby avoiding to need to collect of vast amounts of annotated training data. Cryo-EM has been disruptive in molecular biology and durg discovery. The new methods proposed here will maintain our leadership in this exicting field. Finally, training students in learning and vision is essential; Previous HQP from my group, all residing in Canada, include M Brubaker (Borealis AI), M Norouzi (Google Brain), R Urtasun (Uber ATG), and Leonid Sigal (UBC).