Statistical machine learning for dependent data: symmetry and novel dependence structures

长期目标、方法和HQP 我的长期目标是开发统计和机器学习(ML)方法，通过在模型和计算复杂性之间取得平衡来最大化从数据中提取的信息，并开发描述这种平衡的理论。对称性起着关键作用，起着区分相关和不相关统计信息的作用。方法论和理论方面的动机是在网络分析、进化过程中的应用，如癌症系统发生学、概率编程系统，以及在深度学习和ML中出现的问题。培训HQP，特别是研究生，是当务之急；利用公平的招聘和指导做法，我打算在资助期间培训至少14名HQP的多样化群体。 回顾和最新进展 现代统计学和最大似然法中的许多问题需要在高保真数据模型和执行推理和预测所需的计算复杂性之间找到适当的平衡。统计独立性是并行化和随机优化等高效计算技术的标志，而模型组件之间的太多依赖将导致难以处理的推理方法。相反，没有充分考虑相关性的模型会丢弃关键的统计信息。目前，我们最广泛使用的方法依赖于一小部分简单的依赖结构，而依赖形式较复杂的模型面临困难或不可能的推理目标。我最近的工作解决了网络数据分析和深度学习中的这种权衡问题，使用对称性和所谓的中性向左(NTL)随机过程。 短期目标和影响 在我最近取得的进展的基础上，我有三个短期和有影响力的目标。 1)开发基于新依赖结构的概率模型、推理和应用。我们将强调NTL过程，这是具有未观察到的演变的数据的理想贝叶斯先验。这一目标将产生理论、方法和开源软件，使这些过程对从癌症系统发育到记录联系等广泛领域的从业者有用。 2)创造了基于非传统对称性的高效计算技术的理论和方法。ML的核心计算技术依赖于一组小的对称性，如置换不变性。这个目标将扩展现有的方法并产生新的方法，基于我最近的工作和其他人的工作中出现的对称性。 3)理解条件独立性、对称性和计算之间的关系。条件独立性的统计效用是众所周知的。同样，许多有效的计算方法依赖于条件独立性。对称性是两者之间的天然桥梁。这一目标将扩大我们对条件反射、对称性和计算之间基本关系的理解。

Long-term objective, methods and HQP My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period. Review and recent progress Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes. Short-term objectives and impact Building on my recent progress, I have three short-term and impactful objectives. 1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage. 2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others. 3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.