Evaluation of High-Recall Human-in-the-Loop Information Retrieval Technologies

高查全率信息检索(HRIR)的目标是识别基本上所有与信息需求相关的信息，在这些信息中，丢失或不及时的结果可能会产生严重的法律、政策、健康、社会、安全、国防或财务影响。为了在实践中获得认可，根据管理应用领域的法律、法规、法规、道德或专业标准，HRIR技术必须比目前的做法更有效，而且必须证明更有效。申请人是这一领域的领先专家，他首先认识到机器学习技术从垃圾邮件过滤到电子发现的可转移性，然后为在美国和欧洲的民事诉讼和监管程序中接受这些技术奠定了科学基础。

The objective of high-recall information retrieval (HRIR) is to identify substantially all information relevant to an information need, where the consequences of missing or untimely results may have serious legal, policy, health, social, safety, defence, or financial implications. To find acceptance in practice, HRIR technologies must be more effective—and must be shown to be more effective—than current practice, according to the legal, statutory, regulatory, ethical, or professional standards governing the application domain. The applicant is the leading expert in this field, having first recognized the transferability of machine-learning technologies from spam filtering to electronic discovery, and then having established the scientific foundation for their acceptance in civil litigation and regulatory proceedings, both in the United States and Europe.