Statistical machine learning for dependent data: symmetry and novel dependence structures

长期目标、方法和HQP我的长期目标是开发统计和机器学习（ML）方法，通过在模型和计算复杂性之间取得平衡，最大限度地从数据中提取信息，并发展表征这种平衡的理论。对称性起着关键作用，它将相关和不相关的统计信息区分开来。方法和理论方面受到网络分析、进化过程（如癌症系统发育）、概率规划系统以及深度学习和机器学习中出现的问题的应用的推动。培养HQP，特别是研究生，是重中之重；通过公平的招聘和指导实践，我打算在资助期间培训至少14名HQP的多元化团队。回顾和最新进展现代统计学和机器学习中的许多问题需要在高保真数据模型和执行推理和预测所需的计算复杂性之间找到适当的平衡。统计独立性是并行化和随机优化等高效计算技术的标志，模型组件之间过于依赖将导致难以处理的推理方法。相反，不考虑足够依赖性的模型会丢弃关键的统计信息。目前，我们使用最广泛的方法依赖于一小部分简单的依赖结构，依赖形式更复杂的模型面临难以或不可能的推理目标。我最近的工作是在网络数据分析和深度学习中解决这种权衡，使用对称和所谓的向左中性（NTL）随机过程。短期目标和影响根据我最近的进步，我有三个短期和有影响力的目标。1)基于新的依赖结构开发概率模型、推理和应用。我们将强调NTL过程，它是具有未观察到进化的数据的理想贝叶斯先验。这一目标将产生理论、方法和开源软件，使这些过程对从癌症系统发育到记录链接等广泛领域的从业者有用。2)为基于非传统对称性的高效计算技术创造理论和方法。机器学习的核心计算技术依赖于一小部分对称性，例如置换不变性。这个目标将扩展现有的方法，并基于我最近的工作和其他人的工作中出现的对称性生成新的方法。3)理解条件独立性、对称性和计算之间的关系。条件独立的统计效用是众所周知的。类似地，许多高效的计算方法依赖于条件独立性。对称是两者之间的天然桥梁。这个目标将扩展我们对条件作用、对称性和计算之间的基本关系的理解。

Long-term objective, methods and HQP My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period. Review and recent progress Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes. Short-term objectives and impact Building on my recent progress, I have three short-term and impactful objectives. 1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage. 2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others. 3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.