STTR Phase I: Active Learning System for Audit Selection

该研究项目旨在开发，验证并推向市场的创新，有可能大大提高从欺诈或不合规案件审计的投资回报。在大多数审计检测领域，对案例进行资源密集型评估（如成本高昂的审计）是监控（从而增强）合规性的主要手段。为了优化与安全相关的资源的管理，通常开发统计预测模型来检测不合规的情况。然而，现有的侦查模式发展模式存在着一个根本性缺陷，大大削弱了不遵守情事侦查的效力。用于诱导评分模型的历史数据存在严重偏差-它是从搜索空间中已知具有相对较高的违规可能性的“区域“中提取的。因此，检测模型在应用于检测域的新区域中的不合规时无法产生足够的预测。这一缺陷导致了两个重要后果：（1）检测模型发展缓慢，如果有的话，不遵守行为的变化，并没有有效地发现新的或现有的未知的“口袋”不遵守;（2）新的审计信息只是加强现有的看法，而不是加强现有的知识。从未知区域获取信息以产生更好的检测模型是必要的。该项目的目标是利用机器学习中的智能采样技术，帮助识别特别有用的审计，这将大大改善未来的审计检测和给定成本的收入回收。所提出的技术借鉴了主动学习研究的最新进展，这已被证明产生相当上级模型为一个给定的（审计）收购成本相比，现有的样本采集范例。实证结果表明，在各个行业领域都有令人印象深刻的改善。鉴于审计选择具有重要的独特属性，该项目将现场验证主动学习策略的有效性，为恶意检测域，也许开发定制的新政策，更好地利用审计选择域的属性和目标。我们推测，这些潜在的风险障碍阻碍了目前部署的想法，从主动学习研究，以促进审计选择的做法。从产品的角度来看，这种方法是将主动学习技术（将在第一阶段进行验证）封装为与当前操作系统和业务流程集成的软件系统。该技术可以应用的相关行业领域很广泛，包括税务审计、保险索赔审计、保修欺诈、福利滥用和电子商务欺诈。不遵守的经济影响是巨大的--据估计，1992年未征收的国税局税款为1270亿美元，仅2000年一年，医疗保险就因欺诈和错误而损失了119亿美元。因此，具有成本效益的违规检测可以大大有利于美国经济。

This research project aimes to develop, validate and bring to market an innovation that has the potential to dramatically enhance the return on investment from audit of fraud or non-compliance cases. In most audit detection domains, resource intensive evaluation of cases, such as costly audits, is the principal means of monitoring (and thus enhancing) compliance. To optimize the management of audit-related resources, statistical predictive models are often developed to detect cases of non-compliance. However, there exists a fundamental flaw in the existing paradigm of detection-model development, which significantly undermines the efficacy of non-compliance detection. The historical data used to induce the scoring models is heavily biased - it is drawn from "regions"in the search space that are already known to have relatively higher likelihood of incompliance. As a result, detection models fail to produce adequate predictions when applied to detect non-compliance in new regions of the domains. This flaw results in two important consequences: (1) detection models evolve slowly, if at all, to changes in non-compliance behavior and do not effectively detect new or existing unknown "pockets" of incompliance; and (2) information from new audit merely reinforce existing perceptions rather than enhance current knowledge. It is imperative to acquire information from unknown regions to produce better detection models. The goal of this project is to leverage intelligent sampling techniques from machine learning to help identify particularly informative audits that will substantially improve future audit detection and revenue recovery for a given cost. The proposed technology draws from recent advances in active learning research, which has demonstrated to produce substantially superior models for a given (audit) acquisition cost as compared to the existing sample-acquisition paradigm. Empirical results have shown impressive improvements in a variety of industry domains. Given that audit selection has important unique properties, this project would field validate the efficacy of active learning polices for the audit-detection domain, and perhaps develop customized new policies that better utilize the properties and objectives of the audit selection domain. We conjecture that these potentially risky hurdles have impeded the present deployment of ideas from active learning research to promote audit selection practices. From a product standpoint, the approach is to encapsulate the active learning technology (to be validated in Phase I) as a software system that integrates with current operational systems and business processes. The relevant industry domains to which this technology can be applied are broad and include tax auditing, insurance claims auditing, warranty fraud, benefits abuse, and e-commerce fraud. The economic impact of non-compliance is tremendous - it was estimated that the amount of uncollected IRS taxes in1992 was 127 billion dollars, and that Medicare lost $11.9 billion to fraud and mistakes in 2000 alone. Hence cost-effective detection of noncompliance can substantially benefit the US economy.