Unravel machine learning blackboxes -- A general, effective and performance-guaranteed statistical framework for complex and irregular inference problems in data science

尽管近年来数据分析有了巨大的增长，但数据科学家仍然面临着各种各样的新挑战，包括“黑盒”问题，即机器学习方法可能在经验上有效，但往往缺乏可解释性。这些“黑箱”问题使得解释机器学习结果变得困难，并破坏了对人工智能结果的信任，特别是在医疗保健领域。该研究项目将推进可解释数据科学的基础，并为复杂的不规则统计推断问题开发新的解决方案。该项目将扩大统计和不确定性量化在机器学习和数据科学中的应用。研究成果将纳入课程，以培训下一代统计人员和数据科学家。这一项目将特别重视推动各级教育更广泛地参与统计科学，并将在各种跨学科场所传播研究成果，以促进不同领域的知识综合。更详细地说，该项目将提供新的数学和计算发展来解决不规则推理问题，并将在可解释性方面解开几个机器学习模型中的黑盒。这里，“不规则推理问题”是指现有的规则统计推理条件和大样本理论不适用的高度复杂的问题。研究议程将特别侧重于为离散（数值）或非数值参数问题以及涉及非传统数据的问题（例如，非数值数据），根据三种流行的机器学习模型定制研究：随机森林，深度神经网络和图形网络。该研究计划包括四个子项目：（i）树学习方法和随机森林的不确定性量化;（ii）深度神经网络模型的性能保证架构发现;（iii）生成图形网络的统计推理;以及（iv）解决不规则推理问题的理论发展。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Despite tremendous growth in data analytics in the recent years, data scientists continue to confront a diverse set of emerging challenges, including the “black-box” problems where machine learning methods might be empirically effective but tend to lack interpretability. These “black-box” problems make it difficult to interpret the machine learning results and undermine the trust of the artificial intelligent outcomes, especially in health care domains. This research project will advance the foundations of interpretable data sciences and will develop new solutions for complex irregular statistical inference problems. The project will expand applications of statistics and uncertainty quantification in machine learning and data sciences. The research results will be integrated into course curricula to train the next generation of statisticians and data scientists. This project will pay a particular attention to advancing broader participation in statistical sciences at all educational levels and the research findings will be disseminated in various interdisciplinary venues to bolster knowledge synthesis among different domains. In more detail, this project will provide novel mathematical and computational developments to tackle irregular inference problems and will unravel the black-boxes in several machine learning models in terms of their interpretability. Here, “irregular inference problems” refers to highly complex problems for which the existing regular statistical inference conditions and large sample theories do not apply. The research agenda will focus specifically on providing valid and performance-guaranteed statistical inference for problems concerning discrete (numerical) or non-numerical parameters, and problems involving non-traditional data (e.g., non-numerical data), tailoring the study to the three popular machine learning models: random forests, deep neural networks, and graphical networks. The research initiative includes four subprojects: (i) uncertainty quantification of the tree learning methods and random forests; (ii) performance-guaranteed architecture discovery of deep neural network models; (iii) statistical inference for generative graphical networks; and (iv) theoretical developments for solving irregular inference problems. The results of the projects are expected to improve interpretability of a broader class of machine learning tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.