Statistical machine learning for dependent data: symmetry and novel dependence structures

长期目标、方法和HQP 我的长期目标是开发统计和机器学习（ML）方法，通过在模型和计算复杂性之间取得平衡，最大限度地利用从数据中提取的信息，并开发表征平衡的理论。对称性起着关键的作用，起到分离相关和不相关统计信息的作用。方法和理论方面的动机是网络分析，癌症遗传学，概率编程系统等进化过程中的应用，以及深度学习和ML中出现的问题。培训HQP，特别是研究生，是重中之重;使用公平的招聘和指导实践，我打算在资助期间培训至少14名HQP的多元化团队。 审查和最新进展 现代统计学和机器学习中的许多问题需要在高保真数据模型与执行推理和进行预测所需的计算复杂性之间找到适当的平衡。统计独立性是并行化和随机优化等高效计算技术的标志，模型组件之间的依赖性太大将导致难以处理的推理方法。相反，不考虑足够依赖性的模型会丢弃关键的统计信息。目前，我们最广泛使用的方法依赖于一小部分简单的依赖结构，而具有更复杂依赖形式的模型面临着困难或不可能的推理目标。我最近的工作已经解决了网络数据分析和深度学习中的这种权衡，使用对称性和所谓的中性向左（NTL）随机过程。 短期目标和影响 在我最近取得的进展的基础上，我有三个短期和有效的目标。 1)基于新的依赖结构开发概率模型、推理和应用程序。我们将强调NTL过程，这是理想的贝叶斯先验的数据与未观察到的演变。这一目标将产生理论，方法和开源软件，使这些过程对从癌症遗传学到记录联系等广泛领域的从业者有用。 2)创建基于非传统对称性的高效计算技术的理论和方法。ML中的核心计算技术依赖于一小部分对称性，如置换不变性。这一目标将基于我最近工作和其他人的工作中出现的对称性，扩展现有方法并生成新方法。 3)理解条件独立性、对称性和计算之间的关系。条件独立性的统计效用是众所周知的。类似地，许多有效的计算方法依赖于条件独立性。对称性是两者之间的天然桥梁。这个目标将扩展我们对条件反射、对称性和计算之间基本关系的理解。

Long-term objective, methods and HQP My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period. Review and recent progress Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes. Short-term objectives and impact Building on my recent progress, I have three short-term and impactful objectives. 1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage. 2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others. 3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.