Learning subgroups from data: selective inference and applications

点击翻译按钮获取中文摘要

"Subgroup learning" methods like clustering and regression trees are used to split large, complex data sets into smaller chunks ("subgroups") that are as homogeneous as possible. These methods have been used for important tasks like identifying subgroups of patients that respond differently to an experimental drug, identifying subgroups of tumours that have different biological profiles (leading to targeted treatment strategies), and identifying customers who might be more likely to purchase particular products (leading to targeted advertising strategies). In this proposal, I outline my plans to develop new statistical methodology to determine if identified subgroups in a data set are truly different, and to apply subgroup learning to solve a difficult problem in an area called space-filling designs.  Once we have identified subgroups in a data set, it is natural to want to know whether they are truly different. After all, if the subgroup learning method identified subgroups of tumours that all have the same biological profiles, or subgroups of customers who all have the same purchasing preferences, it would be a massive waste of time and resources to target treatment or advertising strategies to these subgroups. Unfortunately, existing classical statistical methods for testing whether two groups are different are overly optimistic and will almost always claim that the subgroups are different, because they do not account for the double use of the data: once to identify candidate subgroups, and once again to determine whether they are different. To solve this problem, the research program proposes statistical methods that properly account for the double use of the data, when testing for differences in means between subgroups identified using clustering and regression trees.  I will further highlight the untapped potential of subgroup learning methods in a statistical area called space-filling designs, which aims to evenly distribute points throughout a space. These designs are widely used in computer experiments, which use computer models to simulate the effect of input variables on physical systems like weather and sea ice. Space-filling designs are also used to design environmental monitoring networks, like air quality and underwater acoustic monitoring networks. Although spaces are often complex in practice (e.g. networks on coastlines), it is difficult to construct space-filling designs unless the space is simple. I propose a strategy based on an application of a subgroup learning method (hierarchical clustering) that can construct space-filling designs on arbitrarily complex spaces.  In this research program, graduate and undergraduate students will contribute to producing statistical methods that impact the rate of scientific advancement in areas like biology and public health, and become experts in subgroup learning, which will be a valuable asset to their future careers as statisticians or data scientists in academia and industry.