Statistical Machine Learning for Large High Dimensional Complex Data

大型高维复杂数据的统计机器学习

• 批准号: RGPIN-2018-05981
• 负责人: Wang, Steven
• 金额: $ 1.31万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=676946
• 关键词: Statistical; Machine; Learning; Large; Dimensional

1. Machine Learning Theory****** As the core of the modern artificial intelligence, deep learning methods are reshaping our world on a daily basis. However, a general and unified framework still has not been established. Consequently, the powerful deep neural network method still remains as a black box. This lack of theoretical understanding also created difficulties to further improve an effective deep learning method by significantly reducing the computational complexity and extend it to handle more complex situations such as high dimensional non-stationary time series data. ******I would like to establish a theoretical framework and then investigate the theoretical properties of deep learning by building a mathematical and statistical model to describe the behaviors of the deep learning method. In the past, we have been successful in establishing a unified theoretical framework and derive theoretical properties such as convergence and stability for a class of dynamic clustering methods including the famous mean-shift algorithm that has been widely used in image analysis. We will extend our framework to investigate some fundamental issues of machine learning. Our approach is based on statistical mechanics and information theory. Method from physics such as quantum field theory will also be employed. ******2. High Dimensional Statistical Inference ******One of the common challenges for analyzing high dimensional complex data is to reduce the dimensionality efficiently to a manageable level. Common methods include regularization type of method such as Lasso and Bayesian inference which usually reduce the problem into a subspace. Monte Carlo Markov Chain is also an essential tool for Bayesian inference for high dimensional data when the posterior is not analytically trackable. ***We have developed a very effective high dimensional optimization method for searching global optimum for non-convex objective functions. Our method is based on the idea of geodesics or exponential map which is an essential concept in general relativity. We will extend our method to improve the numerical performance of various dimensionality reduction methods and also apply it to high dimensional Bayesian including MCMC. ******3.Statistical Analysis of Brain Data****** We are developing some novel and effective mathematical and statistical models for highly oscillatory non-stationary time series. The new model will take a drastically different approach than most of the traditional spatial-temporal methods which often integrates deterministic and stochastic component in either an additive or certain very restrictive fashion. Our approach will consider the underlying biological mechanism first and then consider other factors that can be best modelled by statistical models. We will also consider a hidden dimension approach which take into account the fact that the information we obtained could be incomplete.

1.作为现代人工智能的核心，深度学习方法每天都在重塑我们的世界。然而，一个普遍和统一的框架仍未建立。因此，强大的深度神经网络方法仍然是一个黑匣子。这种理论理解的缺乏也给进一步改进有效的深度学习方法带来了困难，因为它可以显著降低计算复杂性，并将其扩展到处理更复杂的情况，如高维非平稳时间序列数据。** 我想建立一个理论框架，然后通过建立一个数学和统计模型来描述深度学习方法的行为来研究深度学习的理论属性。在过去，我们已经成功地建立了一个统一的理论框架，并推导出理论性质，如收敛性和稳定性的一类动态聚类方法，包括著名的均值漂移算法，已被广泛应用于图像分析。我们将扩展我们的框架来研究机器学习的一些基本问题。我们的方法是基于统计力学和信息论。也将采用量子场论等物理学方法。2.高维统计推断 * 分析高维复杂数据的常见挑战之一是有效地将维度降低到可管理的水平。常见的方法包括正则化类型的方法，如Lasso和贝叶斯推理，通常将问题简化为子空间。蒙特卡罗马尔可夫链也是一个重要的工具，贝叶斯推断高维数据时，后验是不可追踪的分析。*** 我们开发了一种非常有效的高维优化方法，用于搜索非凸目标函数的全局最优值。我们的方法是基于测地线或指数映射的想法，这是广义相对论中的一个基本概念。 我们将扩展我们的方法来提高各种降维方法的数值性能，并将其应用于包括MCMC在内的高维贝叶斯方法。 * 3.脑数据的统计分析 ** 我们正在开发一些新颖有效的数学和统计模型，用于高度振荡的非平稳时间序列。新的模型将采取一个显着不同的方法，比大多数传统的时空方法，往往集成了确定性和随机成分的添加剂或某些非常限制性的方式。我们的方法将首先考虑潜在的生物学机制，然后考虑可以通过统计模型最好地建模的其他因素。 我们还将考虑隐维方法，该方法考虑到我们获得的信息可能不完整的事实。