CAREER: Content and Cohesion Models, with Applications to Text Summarization and Natural Language Generation

在过去的十年中，概率方法成功地提供了对自然语言文本的分析，进而使广泛的有价值的实际应用成为可能，如机器翻译、问题回答和摘要。尽管取得了这样的成功，但现有的方法存在一个根本的局限性：它们处理每个文档的能力很少或根本没有利用其全球结构的能力。这个项目的目标是为文本的两个基本的、正交的维度--内容和衔接--开发概率模型。一个基于第一个维度内容的模型描述了文本中存在的主题及其组织。第二个维度，衔接，是关于信息在给定的语篇中是如何实现的。这些模型的发展需要能够捕捉复杂文本属性的新的无监督技术，以及主题离散化和语篇语法归纳的新算法。获得好文本的计算度量将在人文和计算机科学的边缘开辟新的研究途径。概率文本模型将构成文本摘要和生成的新方法的基础，这些方法将使在线信息比目前情况下更容易获得。这将极大地影响人们体验多种形式的文本在线信息的方式，包括新闻报道、消费者健康信息和政府文件。学生将通过实践项目、外展项目以及本科生和研究生的课程参与到这项研究中来。

Within the last decade, probabilistic methods have delivered successful analyses of natural language texts that, in turn, have enabled a broad range of valuable and practical applications, such as machine translation, question answering, and summarization. Despite this success, existing methods suffer from a fundamental limitation: they process each document with little or no ability to take advantage of its global structure. All too often, this results in suboptimal performance for the task at hand.The goal of this project is to develop probabilistic models for two fundamental, orthogonal dimensions of text, content and cohesion. A model based on the first dimension, content, describes the topics present in a text and their organization. The second dimension, cohesion, is concerned with how information is realized in a given text. Development of these models requires new unsupervised techniques able to capture complex text properties and novel algorithms for topical discretization and discourse grammar induction.Gaining a computational measure of what constitutes a good text will open new research avenues on the edge of humanities and computer science. Probabilistic text models will form a basis for novel approaches to text summarization and generation that will make on-line information much more accessible than is currently the case. This will substantially affect the way people experience the many forms of textual on-line information, including news reports, consumer health information, and government documents. Students will become involved in this research through hands-on projects, outreach programs, and courses at both the undergraduate and graduate level.