Commonsense Reasoning in Natural Language Processing

自然语言处理(NLP)致力于开发能够以自然语言与人类无缝交互的计算机软件。现在，自然语言处理在我们的日常生活中无处不在，存在于搜索引擎、个人助理和翻译系统等产品中。深度学习的最新突破促进了自然语言处理研究的快速发展，监督学习是主要范式。尽管取得了重大突破，但当前基于DL的NLP模型仍然只能执行狭窄的、通常是特定于领域的任务。他们在涉及学习相对简单的输入-输出映射的任务中表现良好，例如翻译和文本分类，但在涉及缺失信息或需要高级推理的任务中表现不佳。最后，DL模型的有限泛化能力会在与训练分布稍有不同的示例上导致非人性化的错误。这些都是扩大真实世界NLP软件使用的主要障碍。许多系统的生产停滞不前，因为它们还不够健壮，不能在开放域环境中部署。更糟糕的是，其中一些系统被部署，尽管它们的行为不可预测、荒谬和潜在的危险。我研究的长期目标是构建健壮、可靠和准确的NLP系统，该系统可以超越训练分布并处理与人类预期一致的未知输入。我的研究背后的关键假设是，这种能力可以通过赋予NLP模型类似人类的常识和推理能力来实现。在短期内，我将通过3个研究线索来实现这一目标，以解决核心挑战：(1)自动获取(通常未陈述的)常识知识，供机器进行推理；(2)将常识纳入NLP应用程序；以及(3)开发基准和评估协议，以跟踪机器常识方面的进展。我将使用一系列工具和方法，包括深度学习、文本的语言分析、知识库和符号人工智能方法、众包以及从多种形式中学习。通过开发知识获取、整合和推理的核心组件，我将能够训练高性能和健壮的NLP系统，这些系统可以在较少的训练样本下更好地推广。这些组件将改善那些原本是黑箱神经网络的可解释性。这项研究有可能全面提高自然语言处理系统的适用性，包括翻译、对话和摘要。除了对学术界和工业界的直接影响外，拟议的研究计划还将培养新一代自然语言处理领域的一流专家，拥有机器学习、计算机视觉、认知科学和人工智能等邻近研究领域的专业知识。凭借必要的技能和经验，这些专业人士将为加拿大学术界、全球和当地行业做出贡献。

Natural Language Processing (NLP) is concerned with developing computer software that can interact with humans seamlessly in natural language. NLP is now ubiquitous in our everyday life, in products such as search engines, personal assistants, and translation systems. Recent breakthroughs in Deep Learning (DL) contributed to rapid advancement in NLP research, with supervised learning as the main paradigm. Despite significant breakthroughs, current DL-based NLP models are still only able to perform narrow, often domain-specific tasks. They perform well on tasks which involve learning relatively straightforward input-output mappings, such as translation and text classification, but struggle with tasks involving missing information or requiring advanced reasoning. Finally, the limited generalizability of DL models causes unhumanlike errors on examples that even slightly differ from the training distribution. These are major obstacles to broadening the usage of real-world NLP software. The production of many systems is stalled because they are not yet robust enough to be deployed in open-domain settings. Worse still, some of these systems are deployed despite their unpredictable, nonsensical, and potentially dangerous behaviour. The long term goal of my research is to build robust, reliable, and accurate NLP systems that can generalize beyond the training distribution and address unknown inputs consistently with human expectations. The key hypothesis behind my research is that this ability may be achieved by endowing NLP models with humanlike commonsense knowledge and reasoning abilities. In the short term, I will advance towards this goal in 3 research threads addressing core challenges: (1) Automatic acquisition of (often unstated) commonsense knowledge, for machines to reason about; (2) Incorporating commonsense into NLP applications; and (3) Developing benchmarks and evaluation protocols to trace progress on machine commonsense. I will employ a range of tools and approaches including deep learning, linguistic analysis of texts, knowledge bases and symbolic AI methods, crowdsourcing, and learning from multiple modalities. By developing core components for knowledge acquisition, incorporation and reasoning, I will be able to train high-performing and robust NLP systems that can generalize better given fewer training examples. Such components will improve the interpretability of those otherwise black-box neural networks. This research has the potential to improve the applicability of NLP systems across the board, including translation, dialogue, and summarization. Beyond the immediate impact on academia and industry, the proposed research program will train a new generation of first-class experts in NLP, with expertise in adjacent research areas such as machine learning, computer vision, cognitive science, and AI. With the necessary skills and experience, these professionals will contribute to Canadian academia, global and local industry.