EAGER: DCL: SaTC: Enabling Interdisciplinary Collaboration: Efficient Human-in-the-Loop Redaction of Language Development Corpora

在数百名家长、老师和孩子的合作下，研究人员付出了巨大的努力和费用，收集了对话记录，以研究儿童语言发展等主题。对科学最有用的数据是纵向的和自然的，比如在儿童家中定期收集的数据。不幸的是，纵向的，自然主义的语料库最有可能推进知识，可能包含使参与者可识别的信息。因此，自然主义语料库很少与其他研究人员共享，阻碍了科学研究。共享需要仔细的编辑——删除潜在的身份信息。目前，自然语料库对于人工编校来说往往太大，而当前的自动化工具既错过了关键的编校，又过度编校了重要的信息。为了使这些数据能够共享，该项目寻求开发新的编校计算方法。这个项目的目标是开发在非结构化文本数据中识别信息的初步自动化、人工循环编校。首先，为了更好地理解围绕哪些方面的记录可以识别参与者的关键挑战，研究人员正在与社会和行为科学研究人员以及伦理委员会成员进行访谈。根据这些见解，研究人员正在开发新的模型来预测哪些语言可能需要编辑，他们正在设计新的用户交互，以利用人类在编辑决策中的专业知识。对话记录的独特特征需要从自然语言处理、心理学、隐私工程和语言学中汲取语言的新特征。由于自动化方法缺乏人类对会话上下文的洞察力，无法做出复杂的编校决策，因此研究人员正在设计用户界面，总结标记语言或令牌在转录本中纵向出现的方式，使人类编码人员能够快速做出编校决策。作为一个案例研究，研究人员将这些技术应用于语言发展项目，这是一个纵向语料库，记录了100个不同儿童的语言发展。该项目还训练学生进行跨计算科学和社会科学的多学科研究。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

At great effort and expense, and with the cooperation of hundreds of parents, teachers, and children, researchers have collected conversation transcripts to study topics like children's language development. The data most useful for science are longitudinal and naturalistic, such as data collected periodically over time in children's homes. Unfortunately, the longitudinal, naturalistic corpora most likely to advance knowledge may contain information that renders participants identifiable. For this reason, naturalistic corpora are rarely shared with other researchers, hindering science. Sharing requires careful redaction--the removal of potentially identifying information. Currently, naturalistic corpora are often too large for manual redaction, and current automated tools both miss critical redactions and over-redact important information. To enable such data to be shared, this project seeks to develop novel computational methods for redaction.This project's aim is to develop initially automated, human-in-the-loop redaction of identifying information in unstructured text data. First, to better understand key challenges around what aspects of transcripts make participants identifiable, the researchers are conducting interviews with social and behavioral science researchers and members of ethics boards. From these insights, the researchers are developing novel models for predicting what language may need to be redacted and they are designing novel user interactions for leveraging human expertise in redaction decisions. The unique characteristics of conversation transcripts require modeling novel features of language, drawing from natural language processing, psychology, privacy engineering, and linguistics. Because automated methods lack human insights into conversational context for making complex redaction decisions, the researchers are designing user interfaces that summarize how marked language, or tokens, appear longitudinally in transcripts, enabling human coders to quickly make redaction decisions. As a case study, the researchers are applying these techniques to the Language Development Project, a longitudinal corpus of 100 diverse children's development of language. The project is also training students in multidisciplinary research across the computational and social sciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.