GV: EAGER: Innovative Analysis and Visualization Approaches for Understanding Model Uncertainty

这个探索性项目致力于开发一种新的方法来支持人类对复杂模型结构和预测中固有的不确定性的理解。具体地说，该项目的重点是理解与模型预测相关的几种类型的不确定性。当实例空间的区域在训练数据中没有很好地表示时，样本不确定性就会发生，因此预测基于稀疏信息。根据用于构建模型的训练数据，当模型预测不同时，就会发生模型不稳定。当给定的观测值可能具有噪声属性，并且这种输入不确定性导致模型预测中的不确定性时，就会发生预测可变性。开发了新的分析技术来创建表征这三种形式的不确定性的元模型。为了便于用户理解这些多种类型的不确定性在整个模型空间中的性质和分布，新颖的可视化方法在显示空间中表示这些元模型。最后，使用一种新的评估方法来衡量在可视化显示空间中是否以及以何种方式捕捉到元模型的重要特征。这项工作在机器学习和数据可视化领域发展了新的技术。机器学习的贡献包括更强大的构建和分析元模型的方法，这些元模型表征与预测模型相关的多种类型的不确定性。数据可视化研究的重点是表示多值、概率和复杂数据的新方法，从而能够显示模型预测和不确定性的性质和范围。一个跨学科的贡献是开发了一种新的方法，用于在保存重要的模型和元模型特征方面评估模型可视化的质量。该项目的更广泛的影响可以归类为三大类：新的模型构建范式；促进科学合作；以及研究和教育的整合。这些结果可望为进一步研究如何管理代表广泛现象的模型中的不确定性提供基础。该项目奠定了技术基础，有助于促进私人投资机构和应用领域专家之间的新协作，促进广泛的跨学科协作。项目成果将通过项目网站(http://maple.cs.umbc.edu/complexmodels/).)广泛传播最后，通过教学和培训活动，这一研究项目也非常适合向本科生介绍研究的可能性，并将项目主题纳入PIS关于可视化和人工智能的课程。

This exploratory project strives to develop a new approach to support human understanding of the uncertainty that is inherent in the structure and predictions of complex models.Specifically, the focus in the project is on understanding several types of uncertainty that are associated with model predictions.Sample uncertainty occurs when regions of the instance space are not well represented in the training data, and predictions are therefore based on sparse information. Model instability occurs when model predictions vary, depending on the training data that was used to construct the model. Prediction variability occurs when a given observation may have noisy attributes, and this input uncertainty leads to uncertainty in the model's predictions. Novel analytical techniques are developed to create meta-models that characterize these three forms of uncertainty. To facilitate user understanding of the nature and distribution of these multiple types of uncertainty across the model space, novel visualization methods represent these meta-models in a display space. Finally, a novel evaluation methodology is used to measure whether, and in what ways, important characteristics of the meta-models are captured in the visualization display space.This work develops novel techniques in the fields of machine learning and data visualization. Contributions in machine learning include more powerful methods for constructing and analyzing meta-models that characterize multiple types of uncertainty associated with predictive models. Data visualization research focuses on new approaches for representing multi-valued, probabilistic, and complex data, enabling the display of the nature and range of model predictions and uncertainty. An interdisciplinary contribution is the development of a novel methodology for evaluating the quality of model visualizations with respect to the preservation of important model and meta-model characteristics.The broader impacts of this project may be grouped into three major clusters: a new model building paradigm; fostering scientific collaboration; and integrating research and education. The results are expected to provide foundations for further research is management of uncertainty in deriving models representing a wide range of phenomena. This project lays a technical groundwork that can contribute to new collaborations between the PIs and application domain experts, facilitating broad interdisciplinary collaborations. Project results will be widely disseminated via the project web site (http://maple.cs.umbc.edu/complexmodels/). Finally, through teaching and training activities, this research project is also well suited to include the introduction of undergraduates to the possibilities of research and the incorporation of project topics into the PIs' courses on visualization and artificial intelligence.