STTR Phase I: Active Learning System for Audit Selection

该研究项目旨在开发、验证并将一项创新推向市场，该创新有可能大大提高审计欺诈或不合规案例的投资回报。在大多数审计检测领域，对案例进行资源密集的评估，例如昂贵的审计，是监测（并因此加强）遵从性的主要手段。为了优化审计相关资源的管理，通常会开发统计预测模型来检测不合规的情况。然而，现有的检测模型开发范式存在根本性缺陷，严重影响了违规检测的有效性。用于诱导评分模型的历史数据存在严重偏差——它是从搜索空间中的“区域”中提取的，这些“区域”已知存在相对较高的不合规可能性。因此，检测模型在应用于检测域的新区域中的不合规时无法产生足够的预测。这一缺陷导致了两个重要的后果：(1)检测模型对于不合规行为的变化发展缓慢，如果有的话，并且不能有效地检测新的或现有的未知不合规“口袋”；(2)来自新审计的信息只是强化了现有的认知，而不是增强了现有的知识。从未知区域获取信息以产生更好的检测模型势在必行。该项目的目标是利用机器学习中的智能抽样技术来帮助识别特别有用的审计，这将大大改善未来的审计检测和给定成本的收入回收。所提出的技术借鉴了主动学习研究的最新进展，与现有的样本获取范例相比，主动学习研究已经证明，对于给定的（审计）获取成本，它产生了实质上优越的模型。实证结果显示，在许多行业领域都有令人印象深刻的改进。鉴于审计选择具有重要的独特属性，该项目将实地验证审计检测领域主动学习策略的有效性，并可能开发定制的新策略，以更好地利用审计选择领域的属性和目标。我们推测，这些潜在的风险障碍阻碍了目前积极学习研究的思想部署，以促进审计选择实践。从产品的角度来看，该方法是将主动学习技术（在第一阶段进行验证）封装为与当前操作系统和业务流程集成的软件系统。该技术可以应用的相关行业领域非常广泛，包括税务审计、保险索赔审计、保修欺诈、福利滥用和电子商务欺诈。不遵守规定的经济影响是巨大的——据估计，1992年未征收的国税局税款为1270亿美元，仅在2000年，医疗保险就因欺诈和错误损失了119亿美元。因此，具有成本效益的违规检测可以极大地造福美国经济。

This research project aimes to develop, validate and bring to market an innovation that has the potential to dramatically enhance the return on investment from audit of fraud or non-compliance cases. In most audit detection domains, resource intensive evaluation of cases, such as costly audits, is the principal means of monitoring (and thus enhancing) compliance. To optimize the management of audit-related resources, statistical predictive models are often developed to detect cases of non-compliance. However, there exists a fundamental flaw in the existing paradigm of detection-model development, which significantly undermines the efficacy of non-compliance detection. The historical data used to induce the scoring models is heavily biased - it is drawn from "regions"in the search space that are already known to have relatively higher likelihood of incompliance. As a result, detection models fail to produce adequate predictions when applied to detect non-compliance in new regions of the domains. This flaw results in two important consequences: (1) detection models evolve slowly, if at all, to changes in non-compliance behavior and do not effectively detect new or existing unknown "pockets" of incompliance; and (2) information from new audit merely reinforce existing perceptions rather than enhance current knowledge. It is imperative to acquire information from unknown regions to produce better detection models. The goal of this project is to leverage intelligent sampling techniques from machine learning to help identify particularly informative audits that will substantially improve future audit detection and revenue recovery for a given cost. The proposed technology draws from recent advances in active learning research, which has demonstrated to produce substantially superior models for a given (audit) acquisition cost as compared to the existing sample-acquisition paradigm. Empirical results have shown impressive improvements in a variety of industry domains. Given that audit selection has important unique properties, this project would field validate the efficacy of active learning polices for the audit-detection domain, and perhaps develop customized new policies that better utilize the properties and objectives of the audit selection domain. We conjecture that these potentially risky hurdles have impeded the present deployment of ideas from active learning research to promote audit selection practices. From a product standpoint, the approach is to encapsulate the active learning technology (to be validated in Phase I) as a software system that integrates with current operational systems and business processes. The relevant industry domains to which this technology can be applied are broad and include tax auditing, insurance claims auditing, warranty fraud, benefits abuse, and e-commerce fraud. The economic impact of non-compliance is tremendous - it was estimated that the amount of uncollected IRS taxes in1992 was 127 billion dollars, and that Medicare lost $11.9 billion to fraud and mistakes in 2000 alone. Hence cost-effective detection of noncompliance can substantially benefit the US economy.