EAGER: Generating and Understanding Narratives for Dynamic Environments

叙事生成是在动态环境中生成动作的文本描述的过程，如电影、体育赛事和教育节目。对动态环境的在线叙述在从娱乐到培训和教育的广泛背景下都是有益的。例如，成功地解说一段视频将允许盲人和视障人士遵循对理解视频很重要的视觉线索。实现这一目标的关键是将自然语言翻译成计算机可以理解的形式的能力。一个特别的挑战是，不是针对特定选择的领域，而是以适合于适应广泛的自然动态环境的一般方式来做到这一点。该项目探索了解决这些极具挑战性但至关重要的问题的新方向，进行探索性研究，以建立一个领域适应框架的基本组件，该框架学习在人类专家最少的监督下理解和在线生成自然动态环境的叙述。这项研究探索了在线生成叙事的方法，方法是学习环境的自然动态，自动形成模板，并决定何时提及什么。许多自然语言应用都涉及到释义识别和语义理解。这个项目产生的软件和数据可能对自然语言处理中的语义分析有用，并正在提供用于研究目的。这项工作旨在通过广泛的应用程序产生重大的社会影响，包括教育、娱乐和无障碍。叙事生成系统可能有助于视障人士更好地理解互联网上的视频。此外，这样的系统可以帮助广播公司报道新闻或体育赛事，并为不同的用户定制评论。该项目还为本科生和研究生，包括代表不足的群体和少数群体，提供研究和协作工作经验。

Narrative generation is the process of generating textual descriptions of action in dynamic environments such as movies, sports events and educational programs. On-line narration of a dynamic environment is beneficial in a wide range of contexts, from entertainment to training and education. For example, successfully narrating a video would allow blind and visually-impaired people to follow visual cues that are important to understanding the video. Key to attaining this goal is the ability to translate natural language into a form that is understandable by computers. A particular challenge is to do this not for a specifically chosen domain, but in a general way that is suitable for adaptation to a wide range of natural dynamic environments. This project explores new directions to tackle these extremely challenging, yet crucial, issues, undertaking exploratory research towards building essential components of a domain-adaptive framework that learns to understand and generate narratives on-line for natural dynamic environments with minimal supervision by human experts. This research explores methods to generate narratives on-line by learning the natural dynamics of the environment, automatically forming templates, and deciding when and what to mention. Many natural language applications are concerned with recognition of paraphrases and semantic understanding. The software and data resulting from this project are potentially useful for semantic analysis in natural language processing, and is being made available for research purposes. This work is designed for significant social impact through a broad range of applications including educational, entertainment, and accessibility. A narrative generation system could be beneficial to visually-impaired people to better understand videos over the internet. In addition, such a system can help broadcasting companies to report news or sports events with customized commentaries for different users. This project also provides research and collaborative work experience to undergraduate and graduate students including under-represented and minority groups.