Crowd Sourcing Labels From Electronic Medical Records to Enable Biomedical Research

 监督机器学习是一种流行的方法，它使用标记的训练示例来预测未来的结果。 不幸的是，用于生物医学研究的监督机器学习通常受到缺乏标记数据的限制。 当前产生标记数据的方法涉及手动图表审查，其是费力的并且不与数据创建速率成比例。 该项目旨在开发一个框架，通过形成一群临床人员标记器，从电子医疗记录中聚集标记数据集。 这些标记数据集的构建将允许进行以前不可行的新的生物医学研究。 开发临床数据的众包平台存在许多实践和理论挑战。 第一，大众化，大众化 亚马逊的Mechanical Turk等采购平台不适合医疗记录标签，因为HIPAA使临床数据共享存在风险。 其次，临床问题的类型，适合众包没有很好地理解。 第三，临床人群是否能快速准确地制作标签，目前还不清楚。 这些挑战中的每一个都将在一个单独的目标中得到解决。作为该项目的第一个目标，该团队将评估不同的临床众包架构。 该架构必须利用人群的规模，同时最大限度地减少患者信息的暴露。 将考虑使用去识别工具来擦除临床记录，以减少信息泄漏。 使用这种设计，该团队将扩展一个流行的开源众包工具Pybossa，并将其发布给公众。 作为第二个目标，该团队将研究临床预测问题的类型，结构，主题和特异性，以及这些特征如何影响贴标机质量。 最后，该小组将评估的质量和准确性， 收集了两个现有病历审查问题的临床众包数据，以确定平台的实用性。

 DESCRIPTION (provided by applicant): Supervised machine learning is a popular method that uses labeled training examples to predict future outcomes. Unfortunately, supervised machine learning for biomedical research is often limited by a lack of labeled data. Current methods to produce labeled data involve manual chart reviews that are laborious and do not scale with data creation rates. This project aims to develop a framework to crowd source labeled data sets from electronic medical records by forming a crowd of clinical personnel labelers. The construction of these labeled data sets will allow for new biomedical research studies that were previously infeasible to conduct. There are numerous practical and theoretical challenges of developing a crowd sourcing platform for clinical data. First, popular, public crowd sourcing platforms such as Amazon's Mechanical Turk are not suitable for medical record labeling as HIPAA makes clinical data sharing risky. Second, the types of clinical questions that are amenable for crowd sourcing are not well understood. Third, it is unclear if the clinical crowd can produce labels quickly and accurately. Each of these challenges will be addressed in a separate Aim. As the first Aim of this project, the team will evaluate different clinical crowd sourcing architectures. The architecture must leverage the scale of the crowd, while minimizing patient information exposure. De-identification tools will be considered to scrub clinical notes t reduce information leakage. Using this design, the team will extend a popular open source crowd sourcing tool, Pybossa, and release it to the public. As the second Aim, the team will study the type, structure, topic and specificity of clinical prediction questions, and how these characteristics impact labeler quality. Lastly, the team will evaluate the quality and accuracy of collected clinical crowd sourced data on two existing chart review problems to determine the platform's utility.