Optimal statistical exploration and exploitation: theory, methods and applications

点击翻译按钮获取中文摘要

Current challenges to statistics, which offer opportunities for advancing statistical theory and methods, are characterized by complex ethical, theoretical and methodological issues. In many practical applications involving uncertainty, an optimal balance between statistical learning and immediate expected payoff is desirable for the best overall performance of a sequence of decisions. Examples include balancing the collective ethics (exploration) and individual ethics (exploitation) in clinical trials, and high immediate returns (exploitation) and low variability of returns (exploration) in portfolio optimization. Similarly we see the bias-versus-variance tradeoff in statistical inference and fit-versus-complexity tradeoff in statistical modeling. Contributing to my long term research programs that focus on searching for significant and innovative statistical theory and methods to address these tradeoffs and practical socio-economic needs, the proposal specifically centers on two main themes. Theme one focuses on developing and applying new methods of penalized regression for simultaneous shrinkage and variable selection to address the fit-versus-complexity tradeoff. Such methods are at the heart of multicollinearity and high dimensional data modeling. Theme two focuses on dealing with the exploration-versus-exploitation tradeoff, introducing new covariate adjusted response adaptive randomization procedures, systematically examining their theoretical and methodological issues, investigating problems of optimal portfolio selection and introducing new dynamic risk measures. The research is expected to have significant impact on the targeted applications and HQP training. Penalized regression is useful for analyzing gene expression data, response adaptive designs are becoming the new standard for clinical research in desperate medical situations, and dynamic risk measures are important for financial institutions to ensure financial stability. The research actively involves HQPs with clear objectives and methods, trains them effective (independent and collaborative) research skills, communication and career skills, and builds solid background on statistical theory and methodologies in their research fields.