Ensemble subspace, penalty, pretest, and shrinkage strategies for high dimensional data

点击翻译按钮获取中文摘要

There are a host of buzzwords in today's data-centric world. We encounter data in all walks of life, and for analytically- and objectively-minded people, data is crucial to their goals. Making sense of the data and extracting meaningful information from it may not be an easy task. The growth in the size and scope of data sets in a host of disciplines has created a need for innovative statistical strategies for understanding and analyzing such data. A variety of statistical and computational tools are needed to reveal the story that is contained in the data. We define high dimensional data (HDD) as data sets for which the number of predictors are larger than the sample size. The analysis of HDD is an important feature in a host of research fields such as social media, engineering networks, bio-informatics, environmental, and others. The buzzword “Big Data” is nebulously defined, but its problems are real and statisticians play a vital role in this data world. Undoubtedly, overcoming the challenges of HDD is key to successful research in a host of fields. Many organizations are using sophisticated number-crunching, data mining, or Big Data analytics to reveal patterns based on collected information. Clearly, there is an increasing demand for efficient prediction strategies for analyzing HDD. Some examples of HDD that have prompted demand are gene expression arrays, social network modeling, clinical, genetics and phenotypic data.Most of the exiting methods for dealing with HDD begin with model selection for further investigation. Penalized methods are unstable unless very stringent conditions are imposed. This research proposal in HDD focusses on post selection strategies to combat some of the issues inherited in penalized methods. We also propose to investigate ensemble strategy and tuning-parameter free strategy to analyze HDD. Further, I will consider model misspecification problems in HDD and provide a systematic analysis of pretest procedures via divergence theory. Finally, we will develop Bayesian methodology for brain imaging and genetic data. The overarching objective is to provide answers to the question “what are the tools and tricks, pitfalls, applications, challenges and opportunities in HDD analysis”.This proposal emphasizes that statisticians can play a dominant role in solving Big Data problems, and will move statisticians from the cellar of the scientific discovery to the penthouse. The proposed research will provide opportunities for training highly qualified personnel at all levels. The training will be three-fold, methodological, coding/computational, and analysis of data from the real life problems. More public and private sectors are now acknowledging the importance of statistical tools and its critical role in analyzing Big Data. According to a research 4 million jobs may be available globally for Big Data analysis. The proposed research will train individuals for these jobs.