RI: Small: A Data-Driven Framework to Sketch-to-Text Generation

RI：小型：用于生成草图到文本的数据驱动框架

• 批准号: 1524371
• 负责人: Yejin Choi
• 金额: $ 45万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-07-15 至 2019-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1524371&HistoricalAwards=false
• 关键词: RI; Small; Data; Driven; Framework

The project aims to address the limitations of the current natural language generation (NLG) systems by seeking new data-driven approaches to modeling the contextual and creative dimensions of text composition. By taking a large collection of online text as an unstructured database of rhetorical patterns and linguistic creativity, the project develops a statistical generation engine that is capable of composing text with a new level of linguistic creativity and sophistication than what has been previously possible. Formulating sketch-to-text generation as a conceptual framework, the project investigates automatic composition of image captions and product descriptions as application scenarios. The project also explores new possibilities of human-computer collaborative writing, by developing an interactive search-based editor that will assist student writers to learn from a large collection of other people's writings. The technical outcome of the project has the potential to benefit our society in two ways: first, by advancing automatic image captioning for a wide variety of everyday photographs, it can contribute toward equal web access for visually impaired. Second, by enabling interactive search channels over a large-corpus of online writings, it can create new education experiences for training students' writing skills. The project is instrumental for supporting the PI's ongoing efforts in attracting and educating students from underrepresented groups. The proposed research is based on the premise that large-scale online writings, if used correctly, can be an enabling factor for sketch-to-text generation. The project consists of three fundamental research activities. First, the project proposes composition frames and elements as a new conceptual formalism to organize rhetorical patterns as building blocks, and develops unsupervised algorithms to extract them from a large-scale domain-specific corpus. Second, the project develops statistical approaches to differentiate literal language from figurative, with the specific goal of controlling the degree of literalness and creativity in the generated language. Finally, the proposed work designs scalable and robust inference algorithms for composition formulated as constrained optimization. Technical contributions include several unique resources to be shared with the research community, including a new large-scale corpus of image-caption pairs, and the database of learned composition frames and elements.

该项目旨在通过寻求新的数据驱动方法来建模文本组成的上下文和创造性维度，以解决当前自然语言生成（NLG）系统的局限性。通过将大量在线文本作为修辞模式和语言创造力的非结构化数据库，该项目开发了一个统计生成引擎，该引擎能够以比以前更高的语言创造力和复杂性来撰写文本。该项目将草图到文本生成作为一个概念框架，研究图像标题和产品描述的自动合成作为应用场景。该项目还探索了人机协作写作的新可能性，通过开发一个基于搜索的交互式编辑器，帮助学生作家从大量其他人的作品中学习。该项目的技术成果有可能在两个方面造福我们的社会：首先，通过推进各种日常照片的自动图像字幕，它可以为视障人士平等访问网络做出贡献。第二，通过在大量的在线写作语料库上启用交互式搜索通道，可以为训练学生的写作技能创造新的教育体验。该项目有助于支持PI在吸引和教育来自代表性不足群体的学生方面的持续努力。拟议的研究是基于这样的前提下，大规模的在线写作，如果使用正确，可以是一个使能因素的草图到文本生成。该项目包括三项基础研究活动。首先，该项目提出组成框架和元素作为一种新的概念形式主义，以组织修辞模式作为构建块，并开发无监督算法从大规模特定领域的语料库中提取它们。其次，该项目开发了统计方法来区分字面语言和比喻语言，具体目标是控制生成语言的字面性和创造性程度。最后，所提出的工作设计可扩展的和强大的推理算法的组合制定为约束优化。技术贡献包括与研究界共享的几个独特资源，包括一个新的大规模图像字幕对语料库，以及学习构图框架和元素的数据库。