Statistical Machine Learning for Large High Dimensional Complex Data

大型高维复杂数据的统计机器学习

• 批准号: RGPIN-2018-05981
• 负责人: Wang, Steven
• 金额: $ 1.31万
• 依托单位: York University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=668133
• 关键词: Statistical; Machine; Learning; Large; Dimensional

1. Machine Learning Theory****** As the core of the modern artificial intelligence, deep learning methods are reshaping our world on a daily basis. However, a general and unified framework still has not been established. Consequently, the powerful deep neural network method still remains as a black box. This lack of theoretical understanding also created difficulties to further improve an effective deep learning method by significantly reducing the computational complexity and extend it to handle more complex situations such as high dimensional non-stationary time series data. ******I would like to establish a theoretical framework and then investigate the theoretical properties of deep learning by building a mathematical and statistical model to describe the behaviors of the deep learning method. In the past, we have been successful in establishing a unified theoretical framework and derive theoretical properties such as convergence and stability for a class of dynamic clustering methods including the famous mean-shift algorithm that has been widely used in image analysis. We will extend our framework to investigate some fundamental issues of machine learning. Our approach is based on statistical mechanics and information theory. Method from physics such as quantum field theory will also be employed. ******2. High Dimensional Statistical Inference ******One of the common challenges for analyzing high dimensional complex data is to reduce the dimensionality efficiently to a manageable level. Common methods include regularization type of method such as Lasso and Bayesian inference which usually reduce the problem into a subspace. Monte Carlo Markov Chain is also an essential tool for Bayesian inference for high dimensional data when the posterior is not analytically trackable. ***We have developed a very effective high dimensional optimization method for searching global optimum for non-convex objective functions. Our method is based on the idea of geodesics or exponential map which is an essential concept in general relativity. We will extend our method to improve the numerical performance of various dimensionality reduction methods and also apply it to high dimensional Bayesian including MCMC. ******3.Statistical Analysis of Brain Data****** We are developing some novel and effective mathematical and statistical models for highly oscillatory non-stationary time series. The new model will take a drastically different approach than most of the traditional spatial-temporal methods which often integrates deterministic and stochastic component in either an additive or certain very restrictive fashion. Our approach will consider the underlying biological mechanism first and then consider other factors that can be best modelled by statistical models. We will also consider a hidden dimension approach which take into account the fact that the information we obtained could be incomplete.

1. 机器学习理论******作为现代人工智能的核心，深度学习方法每天都在重塑我们的世界。然而，一个通用的、统一的框架还没有建立起来。因此，强大的深度神经网络方法仍然是一个黑盒子。这种理论理解的缺乏也给进一步改进有效的深度学习方法带来了困难，无法显著降低计算复杂度，并将其扩展到处理更复杂的情况，如高维非平稳时间序列数据。******我想建立一个理论框架，然后通过建立一个数学和统计模型来描述深度学习方法的行为来研究深度学习的理论性质。在过去，我们已经成功地建立了一个统一的理论框架，并推导了一类动态聚类方法的理论性质，如收敛性和稳定性，包括在图像分析中广泛使用的著名的mean-shift算法。我们将扩展我们的框架来研究机器学习的一些基本问题。我们的方法是基于统计力学和信息论。物理学的方法，如量子场论也将被采用。* * * * * * 2。高维统计推断******分析高维复杂数据的常见挑战之一是有效地将维数降至可管理的水平。常用的方法包括正则化类型的方法，如Lasso和贝叶斯推理，它们通常将问题缩减到一个子空间。当后验不可分析跟踪时，蒙特卡洛马尔可夫链也是对高维数据进行贝叶斯推理的重要工具。***我们开发了一种非常有效的高维优化方法来搜索非凸目标函数的全局最优。我们的方法是基于广义相对论中的基本概念测地线或指数映射的思想。我们将扩展我们的方法来提高各种降维方法的数值性能，并将其应用于包括MCMC在内的高维贝叶斯。* * * * * * 3。脑数据的统计分析******我们正在为高度振荡的非平稳时间序列开发一些新颖有效的数学和统计模型。新模型将采取一种与大多数传统时空方法截然不同的方法，这些方法通常以一种附加的或某种非常限制的方式集成确定性和随机成分。我们的方法将首先考虑潜在的生物学机制，然后考虑其他因素，这些因素可以最好地通过统计模型建模。我们还将考虑一种隐藏维度方法，该方法考虑到我们获得的信息可能是不完整的。