Development of a general classification framework under the Neyman-Pearson Paradigm, with biomedical and social applications

分类在生物科学、医学、工程、金融和社会科学等各个领域都有广泛的应用。分类的目的是基于标记的训练数据准确地预测新的观察值的类别标签。例如，电子邮件服务提供商需要确定传入的电子邮件是否为垃圾邮件。在不同类型的分类问题中，二元分类是理论、方法和算法发展中最基本、最重要的一类。二元分类中的一个重要问题是如何控制优先级错误类型，要么是类型I错误（将0类数据点错误分类为1类的机会），要么是类型II错误（将1类数据点错误分类为0类的机会）。Neyman-Pearson （NP）分类范式是一种旨在控制一类（或二类）错误并提供理论保证的理论框架。然而，如何用实用的分类算法实现NP范式仍然是一个巨大的挑战。在这项研究中，pi将通过开发新的统计理论、方法、算法和NP范式下的新评估指标来应对这一挑战。该提案的结果将具有广泛的潜在应用，例如降低疾病诊断的假阳性率，提高社交媒体数据对社会事件的预测准确性。项目导师将指导不同背景的研究生和本科生参与拟议的项目，项目成果将在研究生水平的研讨会课程中讲授。为了帮助统计和跨学科研究，pi将把这个项目开发的方法作为开源软件包分发。pi将开发新的统计理论、方法、算法和应用，以控制内曼-皮尔逊（NP）范式下的不对称分类错误。NP范式解决了这样的情况，即用户坚持对类型I错误有特定的限制，同时将类型II错误保持在最低限度。尽管NP范式在假设检验方面已有一个世纪的历史，但直到最近才在分类领域受到重视，其理论和方法尚不完整。为了实现以下四个目标，pi将开发一个通用的NP分类框架，并展示如何将其应用于生物医学和社会科学。在Aim I下，pi将通过探索不同数据结构和样本量的特征依赖和相互作用来开发新的NP分类理论和方法。在Aim II下，pi将设计一个伞形算法，使流行的分类方法适应NP范式。在Aim III下，pi将构建NP版本的受试者工作特征（ROC）曲线：“NP-ROC”，这是一种基于NP分类理论和方法的新评估指标。在目标IV下，pi将把目标I-III中开发的新型NP分类方法应用于大规模生物医学和社会应用。

Classification has broad applications in various fields, including biological sciences, medicine, engineering, finance, and social sciences. The aim of classification is to accurately predict class labels for new observations based on labeled training data. For example, an email service provider needs to decide whether an incoming email is spam. Among different types of classification problems, binary classification is the most basic and important type for theoretical, methodological and algorithmic development. An important question in binary classification is how to control a prioritized type of error, either the type I error (the chance of misclassifying a class 0 data point as class 1) or the type II error (the chance of misclassifying a class 1 data point as class 0). The Neyman-Pearson (NP) classification paradigm is a theoretic framework aiming to control the type I (or type II) error with theoretic guarantee. Yet how to implement the NP paradigm with practical classification algorithms remains a great challenge. In this research, the PIs will tackle this challenge by developing new statistical theory, methods, algorithms, and a novel evaluation metric under the NP paradigm. Results from this proposal will have broad potential applications, such as reducing false positive rates in disease diagnosis and improving prediction accuracy of social events from social media data. The PIs will supervise graduate and undergraduate students of diverse background in the proposed project, and the project outcomes will be taught in graduate-level seminar courses. To aid statistical and interdisciplinary research, the PIs will distribute methods developed in this project as open-source software packages.The PIs will develop new statistical theory, methods, algorithms and applications to control asymmetric classification errors under the Neyman-Pearson (NP) paradigm. The NP paradigm addresses cases where users insist on a specific bound on type I error while keeping type II error to a minimum. Although the NP paradigm has a century-long history in hypothesis testing, until recently it did not receive much attention in the classification area, and its theory and methodologies are as yet incomplete. With the following four aims, the PIs will develop a general NP classification framework and show how it can be applied in the biomedical and social sciences. Under Aim I, the PIs will develop new NP classification theory and methods by exploring feature dependency and interactions for different data structures and sample sizes. Under Aim II, the PIs will design an umbrella algorithm to adapt popular classification methods to the NP paradigm. Under Aim III, the PIs will construct an NP version of Receiver Operating Characteristic (ROC) curves: "NP-ROC", a new evaluation metric based on the NP classification theory and methodologies. Under Aim IV, the PIs will apply the novel NP classification methodologies developed in Aims I-III to large-scale biomedical and social applications.