Collaborative Research: Innovations for Bayesian Tree Ensemble Methodology

跨越许多学科的现代统计分析的一个基本目标是深入了解现实世界过程的行为，以确定变化的重要相关性并获得改进的预测。例如，在市场营销中，统计学家可能有兴趣通过分析消费者交易数据库来了解消费者的购买行为，该数据库包括各种消费者描述符（如年龄、收入水平、地理位置）以及购买金额。统计学家通常会尝试建立一个数学模型来描述消费者描述符和支出金额之间的关系。然而，在这样做的过程中，某些问题对模式的价值和有效性有很大影响。首先，模型的有效性可能在很大程度上依赖于对建模过程性质的先前假设，而这些信息可能难以确定。例如，一个消费者行为模型建立在一个简单的假设上，即高收入水平的消费者总是被期望购买更多，当同时考虑到其他因素时，可能会无意中忽略违反这一假设的微妙之处。其次，有时即使是一个有效的模型也可能是一个非常复杂的对象，以至于提取有意义的信息本身就非常具有挑战性。例如，在建立了一组特定的预测因子作为消费者购买力的重要驱动因素之后，如何最好地衡量它们在模型中的相对重要性仍然是一个关键的兴趣。专注于贝叶斯回归树集成建模的强大而灵活的方法，这个项目的主要推力将是创新这个方法来解决这些和其他建模途径。这种新方法将使从业者能够在假设精益框架中解决他们的研究问题，该框架允许集成模型利用其数据自适应地灵活地合并上下文建模假设。为了极大地提高可解释性，它还将提供自动的、基于不同重要性的信息摘要，以帮助从业者理解和解释可用的描述符信息。除了这些和进一步的方法贡献之外，该项目将开发用于实现该方法的软件，作为免费可用的R包，使从业者能够更容易地在他们的实际工作中利用我们的开发。这就是这个奖项支持的研究生将会有所帮助的地方。研究将集中在贝叶斯集成建模的三个一般创新，以进一步提高其在假设精益框架内从复杂数据中提取意义的能力。第一个贡献将发展具有不同重要性的理论上有效的措施。这些措施将提供计算效率的指数计算，这些指数有意义地衡量预测变量的相对重要性，包括边际和相互作用方面。第二个贡献将提供一种不需要任何单调性先验假设的单调形状约束推理方法。这种多维非参数回归方法将能够发现和估计回归函数的任何和所有单调成分，并且在没有任何约束假设的情况下这样做。第三个贡献将通过为任意响应数据分布（如二分类响应和计数数据）开发BART的泛化，极大地扩展贝叶斯集成建模的适用性。这项主要的技术创新将基于无共轭配方，将BART的范围扩展到许多新的应用领域和问题类型，而不是以前可能的。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

An essential goal of modern statistical analyses across many disciplines is to gain insight into the behavior of real-world processes both to identify important correlates of variation and to obtain improved predictions. For example in marketing, the statistician may be interested in learning the purchasing behavior of consumers from an analysis of a database of consumer transactions that includes various consumer descriptors (e.g. age, income level, geographic location) as well as purchase amounts. The statistician would then typically attempt to build a mathematical model that characterizes the relationship between the consumer descriptors and the expenditure amount. In doing so, however, certain issues bear strongly on the model's value and effectiveness. First, the validity of a model may strongly depend on prior assumptions about the nature of the modeled process, information that can be difficult to ascertain. For instance, a consumer behavior model which builds in a simple assumption that consumers with higher income levels are always expected to purchase more, may be inadvertently ignoring subtleties that violate this assumption when other factors are simultaneously taken into account. Second, sometimes even a valid and effective model may be such a complicated object that the extraction of meaningful information can itself be very challenging. For example, after establishing particular set of predictors as important drivers of consumer purchasing power, it will still be of key interest how to best measure their relative importance in the model. Focusing on the powerful and flexible approach of Bayesian regression tree ensemble modeling, the main thrust of this project will be to innovate this methodology to address these and other modeling avenues. This new methodology will enable practitioners to address their research questions in an assumption-lean framework that allows the ensemble models to make use of their data to adaptively and flexibly incorporate contextual modeling assumptions. To greatly enhance interpretability, it will also provide automatic, information based summaries of variable importance to help the practitioner understand and interpret the available descriptor information. In addition to these and further methodological contributions, the project will develop software for the implementation of this methodology as a freely available R package, enabling practitioners to more easily leverage our developments in their practical work. This is where the graduate student supported by this award will help. The research will focus on three general innovations to Bayesian ensemble modeling to further enhance its ability to extract meaning from complex data within an assumption lean framework. The first contribution will develop theoretically valid measures of variable importance. These measures will provide computationally efficient calculation of indices which meaningfully gauge the relative importance of predictor variables, both marginally and in terms of interactions. The second contribution will provide an approach to monotone shape constrained inference which does not require any prior assumption of monotonicity. This multidimensional nonparametric regression approach will enable the discovery and estimation of any and all the monotone components of the regression function, and to do so with no constraint assumptions whatsoever. The third contribution will vastly extend the applicability of Bayesian ensemble modeling by developing a generalization of BART for arbitrary response data distributions, such as dichotomous responses and count data. This major technical innovation will be based on a conjugacy-free formulation that will extend the reach of BART to many new application areas and problem types than were previously possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.