STTR Phase I: Active Learning System for Audit Selection

这一研究项目旨在开发、验证一项创新并将其推向市场，该创新有可能显著提高对欺诈或违规案件的审计投资回报。在大多数审计检测领域，对案件进行资源密集型评估，例如成本高昂的审计，是监测(从而加强)遵从性的主要手段。为了优化审计相关资源的管理，经常开发统计预测模型来检测不遵守情况。然而，现有的检测模型开发范式存在一个根本性的缺陷，这大大削弱了不合规检测的有效性。用于建立评分模型的历史数据存在严重偏差--它来自搜索空间中已知的不符合条件的可能性相对较高的“区域”。因此，当应用检测模型来检测域的新区域中的不合规时，检测模型无法产生足够的预测。这一缺陷导致两个重要后果：(1)检测模型对违规行为的变化发展缓慢(如果有的话)，并且不能有效地检测新的或现有的未知不合规“口袋”；(2)来自新审计的信息只是加强了现有的看法，而不是增强当前的知识。从未知区域获取信息以产生更好的检测模型是当务之急。该项目的目标是利用机器学习的智能抽样技术来帮助确定特别信息丰富的审计，这将大大改进未来的审计检测和对给定成本的收入回收。拟议的技术借鉴了主动学习研究的最新进展，与现有的样本获取范式相比，主动学习研究已经证明，对于给定的(审计)获取成本，可以产生显著优越的模型。经验结果表明，在各种行业领域都有令人印象深刻的改进。鉴于审计选择具有重要的独特属性，该项目将实地验证主动学习策略对审计检测领域的有效性，并可能制定定制的新策略，以更好地利用审计选择领域的属性和目标。我们推测，这些潜在的风险障碍阻碍了目前积极学习研究的想法的部署，以促进审计选择实践。从产品的角度来看，方法是将主动学习技术(将在阶段I中验证)封装为与当前操作系统和业务流程集成的软件系统。这项技术可以应用到的相关行业领域很广泛，包括税务审计、保险索赔审计、保修欺诈、福利滥用和电子商务欺诈。不遵守规定的经济影响是巨大的--据估计，1992年美国国税局未征收的税款为1270亿美元，仅在2000年，医疗保险就因欺诈和错误而损失了119亿美元。因此，经济高效地检测违规行为可以极大地惠及美国经济。

This research project aimes to develop, validate and bring to market an innovation that has the potential to dramatically enhance the return on investment from audit of fraud or non-compliance cases. In most audit detection domains, resource intensive evaluation of cases, such as costly audits, is the principal means of monitoring (and thus enhancing) compliance. To optimize the management of audit-related resources, statistical predictive models are often developed to detect cases of non-compliance. However, there exists a fundamental flaw in the existing paradigm of detection-model development, which significantly undermines the efficacy of non-compliance detection. The historical data used to induce the scoring models is heavily biased - it is drawn from "regions"in the search space that are already known to have relatively higher likelihood of incompliance. As a result, detection models fail to produce adequate predictions when applied to detect non-compliance in new regions of the domains. This flaw results in two important consequences: (1) detection models evolve slowly, if at all, to changes in non-compliance behavior and do not effectively detect new or existing unknown "pockets" of incompliance; and (2) information from new audit merely reinforce existing perceptions rather than enhance current knowledge. It is imperative to acquire information from unknown regions to produce better detection models. The goal of this project is to leverage intelligent sampling techniques from machine learning to help identify particularly informative audits that will substantially improve future audit detection and revenue recovery for a given cost. The proposed technology draws from recent advances in active learning research, which has demonstrated to produce substantially superior models for a given (audit) acquisition cost as compared to the existing sample-acquisition paradigm. Empirical results have shown impressive improvements in a variety of industry domains. Given that audit selection has important unique properties, this project would field validate the efficacy of active learning polices for the audit-detection domain, and perhaps develop customized new policies that better utilize the properties and objectives of the audit selection domain. We conjecture that these potentially risky hurdles have impeded the present deployment of ideas from active learning research to promote audit selection practices. From a product standpoint, the approach is to encapsulate the active learning technology (to be validated in Phase I) as a software system that integrates with current operational systems and business processes. The relevant industry domains to which this technology can be applied are broad and include tax auditing, insurance claims auditing, warranty fraud, benefits abuse, and e-commerce fraud. The economic impact of non-compliance is tremendous - it was estimated that the amount of uncollected IRS taxes in1992 was 127 billion dollars, and that Medicare lost $11.9 billion to fraud and mistakes in 2000 alone. Hence cost-effective detection of noncompliance can substantially benefit the US economy.