Statistical machine learning for dependent data: symmetry and novel dependence structures

点击翻译按钮获取中文摘要

Long-term objective, methods and HQP My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period. Review and recent progress Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes. Short-term objectives and impact Building on my recent progress, I have three short-term and impactful objectives. 1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage. 2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others. 3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.