III: Medium: Development and Evaluation of Search Technology for Discovery of Evidence in Civil Litigation

美国的民事诉讼制度是商业和个人纠纷的最终仲裁者。 在这一制度下，原告和被告都有权要求对方提供相关证据。 虽然数字记录似乎比旧的纸质记录更容易找到，但这些记录的数量、多样性和可能位置的快速增长实际上使在数字干草堆中找到众所周知的针变得更加困难。 由此造成的发现和交换相关证据的费用迅速增加，如果不加以控制， 这引起了人们对诉诸司法的关切。因此，我们迫切需要一种确实准确和具成本效益的技术，以支持“电子发现”相关记录。马里兰州大学的奥尔德和同事们正在开发一种技术，可以在几分钟内自动判断一个人一生中无法检查的文件的响应性。 这些技术使用“半监督学习”算法来“训练”软件，以复制人们对代表性示例所做的各种决策。使用有限人口注释，一个新的框架，将学习与评估相结合，正在开发新的方法，以实现和衡量任何特定水平的人类努力的最高可能的有效性。 这些学习方法借鉴了丰富的方法来表示数字结构化文档和扫描纸张的内容。 正在制定严格评估由此产生的自动审查技术的有效性的措施，以支持法律的专业人员和法院关于使用哪种方法的决定，并帮助开发人员进一步改进其算法。 法律的系统要求的技术，其有效性已被证明是代表什么是实际预期在一个真实的案件的集合。 出于这个原因，这个项目正在与美国国家标准与技术研究所的文本检索会议（TREC）合作创建真实的世界基准。 预计该项目的成果将有助于通过为法律的和技术利益攸关方举办讲习班，并通过大学课程培养下一代律师和信息专业人员运用这些新能力，从而形成专业做法。 这一努力所产生的“电子发现”技术很可能广泛适用于法律实践以外的领域，包括编写科学文献的系统评论、学术界对数字档案的访问以及政府对公民公共信息请求的回应。 更多信息请访问http://ediscovery.umiacs.umd.edu。

The civil litigation system of the United States serves as the ultimate arbiter for commercial and personal disputes. Under this system, plaintiffs and defendants are entitled to request relevant evidence from each other. Although digital records seem easier to find than their older paper counterparts, rapid growth in the volume, diversity, and possible locations of these records has actually made it harder to find the proverbial needles within the digital haystacks. The resulting rapid increase in the cost of discovery and exchange of relevant evidence, if left unchecked, raises concerns about access to justice. Hence, there is an urgent need for demonstrably accurate and cost-effective technologies to support "e-discovery" of the relevant records.Professor Douglas W. Oard and colleagues of the University of Maryland are developing techniques to automatically decide within minutes the responsiveness of more documents than one person could examine in a lifetime. These techniques use "semi-supervised learning" algorithms for "training" the software to replicate the kinds of decisions that people make on representative examples. Using Finite Population Annotation, a new framework for integrating learning with evaluation, novel methods are being developed to achieve and measure the highest possible effectiveness for any specified level of human effort. These learning methods draw on rich approaches to representing the content of both born-digital structured documents and scanned paper. Measures for rigorously assessing the effectiveness of the resulting automated review techniques are being developed both to support decisions by legal professionals and by the courts about which methods to use, and to help developers further improve their algorithms. The legal system demands technology whose effectiveness has been demonstrated on collections that are representative of what is actually expected in a real case. For that reason, this project is creating real world benchmarks in collaboration with the National Institute of Standards and Technology's Text Retrieval Conference (TREC). The project's results are expected to help to shape professional practice through workshops for legal and technical stakeholders, and through university courses to prepare the next generation of attorneys and information professionals to employ these new capabilities. "E-discovery" technologies resulting from this effort are likely to be broadly applicable in domains beyond the law practice, including preparation of systematic reviews of scientific literature, scholarly access to digital archives, and government responses to public information requests from citizens. Additional information is available at http://ediscovery.umiacs.umd.edu.