Model Evaluation in Modern Predictive Regimes: Case Influence and Model Complexity

现代预测机制中的模型评估:案例影响和模型复杂性

基本信息

  • 批准号:
    2015490
  • 负责人:
  • 金额:
    $ 25万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2020
  • 资助国家:
    美国
  • 起止时间:
    2020-07-01 至 2024-06-30
  • 项目状态:
    已结题

项目摘要

Models are an integral component of scientific inquiries, and they can be used as effective devices in many practical areas as engineering, commerce, and governance. There have been remarkable advances in the way statistical models are defined from data for prediction or for accurate descriptions of the world we observe. While complex predictive models have routinely emerged, our understanding of these models and methods for their evaluation have been lagging. Appropriate tools for assessing models and recognizing their deficiencies, and proper ways to account for their complexity are in great need. To fill these gaps, the project aims to develop methodologies and computational tools for assessment of case influence on general models in predictive settings. And it will extend the notion of model complexity to general prediction rules for model comparison and calibration by using the overall model sensitivity to data perturbation. Results from this research will bring great benefit not only to science and engineering through the practice of refined statistical modeling, but also to society at large through applications. In particular, the project will have practical utility in outlier detection for many scientific applications, fraud/threat detection and prevention for many business applications, and detection of adversarial attacks for artificial intelligence applications. Moreover, it will advance our understanding of modern algorithmic models such as deep learning through the research on model complexity and foster interdisciplinary research. The project will provide research training opportunities for graduate students. Computational tools developed will be distributed as open-source software.To characterize the sensitivity of a predictive model to data, the PIs will develop novel approaches to case influence assessment for general modeling procedures, encompassing many modern statistical learning techniques for classification and regression. Extending case deletion statistics and case influence graph in linear regression, the project will offer a variety of new case influence measures for classification in particular. In addition, the PIs will develop efficient computational algorithms for evaluating those case influence measures by utilizing a homotopy technique to relate two modeling problems with the original data and perturbed data under various perturbation schemes. Further, the project will examine model complexity through the lens of model sensitivity to data perturbation considered in case influence assessment and extend the notion of model degrees of freedom to general modeling procedures including large-margin classifiers. This extension will be based on the relation between expected optimism and model complexity in the risk estimation framework where model sensitivities to perturbation of individual cases can be linked to the conditional expected optimism.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
模型是科学研究的一个组成部分,它们可以在工程、商业和治理等许多实际领域中用作有效的设备。从数据中定义统计模型的方式已经取得了显着的进步,用于预测或准确描述我们观察到的世界。虽然复杂的预测模型经常出现,但我们对这些模型及其评估方法的理解一直滞后。非常需要评估模型和认识其缺陷的适当工具,以及说明其复杂性的适当方法。为了填补这些空白,该项目旨在开发方法和计算工具,用于评估预测环境中病例对一般模型的影响。它还将模型复杂性的概念扩展到一般的预测规则,通过使用整体模型对数据扰动的敏感性来进行模型比较和校准。本研究的成果不仅将通过精细化统计建模的实践为科学和工程带来巨大的好处,而且将通过应用为整个社会带来巨大的好处。特别是,该项目将在许多科学应用的异常值检测、许多商业应用的欺诈/威胁检测和预防以及人工智能应用的对抗性攻击检测方面具有实用价值。此外,它还将通过对模型复杂性的研究,促进我们对深度学习等现代算法模型的理解,并促进跨学科研究。该项目将为研究生提供研究培训机会。为了表征预测模型对数据的敏感性,PI将开发新的方法来评估一般建模程序的案例影响,包括许多现代统计学习技术,用于分类和回归。在线性回归中扩展案例删除统计和案例影响图,该项目将提供各种新的案例影响度量,特别是分类。此外,PI将开发有效的计算算法,通过利用同伦技术将两个建模问题与原始数据和各种扰动方案下的扰动数据联系起来,以评估这些情况的影响措施。此外,该项目将通过案例影响评估中考虑的模型对数据扰动的敏感性的透镜来检查模型的复杂性,并将模型自由度的概念扩展到包括大裕度分类器在内的一般建模程序。这一扩展将基于风险评估框架中的预期乐观和模型复杂性之间的关系,其中模型对个别情况扰动的敏感性可以与有条件的预期乐观联系起来。该奖项反映了NSF的法定使命,并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。

项目成果

期刊论文数量(6)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
The Geometry of Nonlinear Embeddings in Kernel Discriminant Analysis
An algorithmic view of l2-regularization and some path-following algorithms
l2-正则化和一些路径跟踪算法的算法视图
A convex optimization formulation for multivariate regression
多元回归的凸优化公式
On the consistent estimation of optimal Receiver Operating Characteristic (ROC) curve
最优受试者工作特征(ROC)曲线的一致性估计
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Yoonkyung Lee其他文献

Dihydrostilbenes and flavonoids from whole plants of emJacobaea vulgaris/em
来自普通鬼针草全株的二氢芪类和黄酮类化合物
  • DOI:
    10.1016/j.phytochem.2024.114107
  • 发表时间:
    2024-06-01
  • 期刊:
  • 影响因子:
    3.400
  • 作者:
    Shinae Lee;Min-Gyung Son;Young-Mi Kim;Chae-Yeong An;Hyun Ji Kim;Piseth Nhoek;Pisey Pel;Hongic Won;Yoonkyung Lee;Narae Yun;Jin-Hyub Paik;Badamtsetseg Bazarragchaa;Hyun Woo Kim;Young Hee Choi;Won Keun Oh;Chang Hoon Lee;Young-Won Chin
  • 通讯作者:
    Young-Won Chin
light-The relationship between sleep and innate immunity
光-睡眠与先天免疫的关系
  • DOI:
  • 发表时间:
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Yoonkyung Lee;Kyung
  • 通讯作者:
    Kyung
Articulating Inequality in the Candlelight Protests of 2016–2017
阐明 2016-2017 年烛光抗议中的不平等
  • DOI:
    10.25024/kj.2019.59.1.16
  • 发表时间:
    2019
  • 期刊:
  • 影响因子:
    0.4
  • 作者:
    Yoonkyung Lee
  • 通讯作者:
    Yoonkyung Lee
Militants or Partisans: Labor Unions and Democratic Politics in Korea and Taiwan
激进分子还是游击队:韩国和台湾的工会与民主政治
  • DOI:
    10.11126/stanford/9780804775373.001.0001
  • 发表时间:
    2011
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Yoonkyung Lee
  • 通讯作者:
    Yoonkyung Lee
Cold War Undercurrents: The Extreme-Right Variants in East Asia*
冷战暗流:东亚的极右变种*
  • DOI:
    10.1177/00323292211033080
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    2.4
  • 作者:
    Yoonkyung Lee
  • 通讯作者:
    Yoonkyung Lee

Yoonkyung Lee的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Yoonkyung Lee', 18)}}的其他基金

Nonlinear Dimension Reduction Methods
非线性降维方法
  • 批准号:
    1513566
  • 财政年份:
    2015
  • 资助金额:
    $ 25万
  • 项目类别:
    Standard Grant

相似国自然基金

基于重要农地保护LESA(Land Evaluation and Site Assessment)体系思想的高标准基本农田建设研究
  • 批准号:
    41340011
  • 批准年份:
    2013
  • 资助金额:
    20.0 万元
  • 项目类别:
    专项基金项目

相似海外基金

The re-evaluation of modern traditional townscapes based on the analysis of reconstructions after conflagration
基于火灾重建分析的现代传统城镇景观再评价
  • 批准号:
    21H04579
  • 财政年份:
    2021
  • 资助金额:
    $ 25万
  • 项目类别:
    Grant-in-Aid for Scientific Research (A)
Reviewing the Genjohenko Concept of Cultural Properties focusing on Contemporaneity in the Value Evaluation of Modern Architectural Heritage
回顾现代建筑遗产价值评价中注重当代性的文化财产观
  • 批准号:
    21K04474
  • 财政年份:
    2021
  • 资助金额:
    $ 25万
  • 项目类别:
    Grant-in-Aid for Scientific Research (C)
Elucidation of the structural mechanism of the seismic walls with wooden substrates and plaster work in modern wooden buildings and earthquake damage evaluation concerning them
现代木结构建筑中木质基材抗震墙和抹灰结构机理的阐明及其震害评价
  • 批准号:
    21K14287
  • 财政年份:
    2021
  • 资助金额:
    $ 25万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Molecular and clinical evaluation of the glioma patient experience to anticipate modern outcomes and guide patient care
对神经胶质瘤患者经历进行分子和临床评估,以预测现代结果并指导患者护理
  • 批准号:
    10472504
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
Analysis and Evaluation of Late Modern Japanese Colloquial Materials from Quantitative Perspectives
近代后期日语口语材料的定量分析与评价
  • 批准号:
    18K12402
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Molecular and clinical evaluation of the glioma patient experience to anticipate modern outcomes and guide patient care
对神经胶质瘤患者经历进行分子和临床评估,以预测现代结果并指导患者护理
  • 批准号:
    10226954
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
Molecular and clinical evaluation of the glioma patient experience to anticipate modern outcomes and guide patient care
对神经胶质瘤患者经历进行分子和临床评估,以预测现代结果并指导患者护理
  • 批准号:
    9759883
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
Molecular and clinical evaluation of the glioma patient experience to anticipate modern outcomes and guide patient care
对神经胶质瘤患者经历进行分子和临床评估,以预测现代结果并指导患者护理
  • 批准号:
    10675604
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
A study on the historical development of evaluation trade and its structural analysis in the early modern Ryukyu Kingdom
近代早期琉球王国估价贸易的历史发展及其结构分析研究
  • 批准号:
    18K12492
  • 财政年份:
    2018
  • 资助金额:
    $ 25万
  • 项目类别:
    Grant-in-Aid for Early-Career Scientists
Technical Feasibility and Evaluation Study of Modern Computational Chemistry Methods for the Study of High Atomic Number Materials
现代计算化学方法研究高原子序数材料的技术可行性及评价研究
  • 批准号:
    ST/R000026/1
  • 财政年份:
    2017
  • 资助金额:
    $ 25万
  • 项目类别:
    Research Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了