Statistical learning algorithms for high-dimensional non-normally distributed data

点击翻译按钮获取中文摘要

Computational methods to reflect a variety of data are continuing to improve. This proposal suggests three main threads of addressing issues with modelling and discovering patterns in data with statistical learning techniques.The first thread of research considers binary data bases. From records of receipts keeping track of what customers purchase to observations indicating the factors involved in an accident, binary data bases are both common, and can be very large. Summarizing associations in these data bases is a useful task for a variety of reasons. Many possible associations exist and comparing them to each other is important. Numerically comparing these associations is particularly valuable as this can be automated by the computer in large scales. However, the choice of numerical summary is important. This proposal suggests methods of improving numerical summaries of associations in binary data to elucidate patterns. One of these methods is on how to summarize data when some of the binary variables are actually elements of a categorical variable, and another is to consider how noteworthy these values are in light of the distribution of the data. A second thread of the proposal addresses the complexity of models. Although very complex models can accurately model some data, this is undesirable for a variety of reasons including interpretability, robustness, and computational challenges. Some complex models can be simplified by considering when certain parameters, quantities which define a model, are constrained to be the same as other parameters of the model. This relates whatever the parameters represent, reduces the number of estimates the computer requires, and make the model easier to interpret. This proposal suggests explores a recently proposed method of discovering these constraints for a variety of statistical models. The final thread of this proposal addresses the realistic issue of the assumptions made when modelling what are often considered to be "continuous" variables. These data are often modeled as truly continuous, following a particular distribution (the normal distribution), or both. In this thread, more flexible assumptions are considered and accommodates the situation that data representing continuous variables are actually only known up to a limited precision which can influence results. The exploration will determine in which scenarios this limited precision matters and how accurate answers are when accounting for the limited precision and less stringent assumptions. All of these issues will be addressed as highly qualified personnel develop and apply new skills, trained in the analysis of realistic, sometimes inconvenient, and big data. This is a skillset that has been identified as a "talent gap" within Canada and will be addressed with this proposal.