Development of a general classification framework under the Neyman-Pearson Paradigm, with biomedical and social applications

在内曼-皮尔逊范式下开发通用分类框架，并具有生物医学和社会应用

• 批准号: 1613338
• 负责人: Xin Tong
• 金额: $ 12万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-08-15 至 2019-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1613338&HistoricalAwards=false
• 关键词: Development; general; classification; framework; under

Classification has broad applications in various fields, including biological sciences, medicine, engineering, finance, and social sciences. The aim of classification is to accurately predict class labels for new observations based on labeled training data. For example, an email service provider needs to decide whether an incoming email is spam. Among different types of classification problems, binary classification is the most basic and important type for theoretical, methodological and algorithmic development. An important question in binary classification is how to control a prioritized type of error, either the type I error (the chance of misclassifying a class 0 data point as class 1) or the type II error (the chance of misclassifying a class 1 data point as class 0). The Neyman-Pearson (NP) classification paradigm is a theoretic framework aiming to control the type I (or type II) error with theoretic guarantee. Yet how to implement the NP paradigm with practical classification algorithms remains a great challenge. In this research, the PIs will tackle this challenge by developing new statistical theory, methods, algorithms, and a novel evaluation metric under the NP paradigm. Results from this proposal will have broad potential applications, such as reducing false positive rates in disease diagnosis and improving prediction accuracy of social events from social media data. The PIs will supervise graduate and undergraduate students of diverse background in the proposed project, and the project outcomes will be taught in graduate-level seminar courses. To aid statistical and interdisciplinary research, the PIs will distribute methods developed in this project as open-source software packages.The PIs will develop new statistical theory, methods, algorithms and applications to control asymmetric classification errors under the Neyman-Pearson (NP) paradigm. The NP paradigm addresses cases where users insist on a specific bound on type I error while keeping type II error to a minimum. Although the NP paradigm has a century-long history in hypothesis testing, until recently it did not receive much attention in the classification area, and its theory and methodologies are as yet incomplete. With the following four aims, the PIs will develop a general NP classification framework and show how it can be applied in the biomedical and social sciences. Under Aim I, the PIs will develop new NP classification theory and methods by exploring feature dependency and interactions for different data structures and sample sizes. Under Aim II, the PIs will design an umbrella algorithm to adapt popular classification methods to the NP paradigm. Under Aim III, the PIs will construct an NP version of Receiver Operating Characteristic (ROC) curves: "NP-ROC", a new evaluation metric based on the NP classification theory and methodologies. Under Aim IV, the PIs will apply the novel NP classification methodologies developed in Aims I-III to large-scale biomedical and social applications.

分类在生物科学、医学、工程、金融、社会科学等领域有着广泛的应用。分类的目的是根据标注的训练数据准确地预测新观测的类别标签。例如，电子邮件服务提供商需要确定传入的电子邮件是否为垃圾邮件。在不同类型的分类问题中，二分类是理论、方法和算法发展中最基本、最重要的类型。二进制分类中的一个重要问题是如何控制区分优先级的错误类型，要么是I类错误(将0类数据点错误分类为1类的可能性)，要么是II类错误(将1类数据点错误分类为0类的可能性)。Neyman-Pearson(NP)分类范式是一种理论框架，旨在为控制第一类(或第二类)错误提供理论保障。然而，如何用实用的分类算法来实现NP范型仍然是一个巨大的挑战。在这项研究中，PI将通过发展新的统计理论、方法、算法和NP范式下的新评估指标来应对这一挑战。这一建议的结果将具有广泛的潜在应用，如降低疾病诊断中的假阳性率，提高从社交媒体数据预测社交事件的准确性。在拟议的项目中，项目督导将指导不同背景的研究生和本科生，项目成果将在研究生级别的研讨会课程中教授。为协助统计和跨学科研究，统计研究所将以开放源码软件的形式分发在这项计划中开发的方法，并将发展新的统计理论、方法、算法和应用程序，以控制奈曼-皮尔逊(NP)模式下的不对称分类错误。NP范式解决了这样的情况：用户坚持类型I错误的特定界限，同时将类型II错误保持在最低限度。尽管NP范式在假设检验方面已经有一个世纪的历史，但直到最近才在分类领域受到足够的重视，其理论和方法也仍然不完善。在以下四个目标下，个人信息系统将开发一个通用的NP分类框架，并展示如何将其应用于生物医学和社会科学。在目标I下，个人信息系统将通过探索不同数据结构和样本大小的特征依赖和相互作用来开发新的NP分类理论和方法。在AIM II下，个人信息系统将设计一个伞形算法，使流行的分类方法适应NP范式。在AIM III下，私营部门将构建接收器工作特性(ROC)曲线的NP版本：“NP-ROC”，这是一种基于NP分类理论和方法的新评估指标。在AIM IV下，PIS将把AIMS I-III中开发的新NP分类方法应用于大规模生物医学和社会应用。