Commonsense Reasoning in Natural Language Processing

点击翻译按钮获取中文摘要

Natural Language Processing (NLP) is concerned with developing computer software that can interact with humans seamlessly in natural language. NLP is now ubiquitous in our everyday life, in products such as search engines, personal assistants, and translation systems. Recent breakthroughs in Deep Learning (DL) contributed to rapid advancement in NLP research, with supervised learning as the main paradigm. Despite significant breakthroughs, current DL-based NLP models are still only able to perform narrow, often domain-specific tasks. They perform well on tasks which involve learning relatively straightforward input-output mappings, such as translation and text classification, but struggle with tasks involving missing information or requiring advanced reasoning. Finally, the limited generalizability of DL models causes unhumanlike errors on examples that even slightly differ from the training distribution. These are major obstacles to broadening the usage of real-world NLP software. The production of many systems is stalled because they are not yet robust enough to be deployed in open-domain settings. Worse still, some of these systems are deployed despite their unpredictable, nonsensical, and potentially dangerous behaviour. The long term goal of my research is to build robust, reliable, and accurate NLP systems that can generalize beyond the training distribution and address unknown inputs consistently with human expectations. The key hypothesis behind my research is that this ability may be achieved by endowing NLP models with humanlike commonsense knowledge and reasoning abilities. In the short term, I will advance towards this goal in 3 research threads addressing core challenges: (1) Automatic acquisition of (often unstated) commonsense knowledge, for machines to reason about; (2) Incorporating commonsense into NLP applications; and (3) Developing benchmarks and evaluation protocols to trace progress on machine commonsense. I will employ a range of tools and approaches including deep learning, linguistic analysis of texts, knowledge bases and symbolic AI methods, crowdsourcing, and learning from multiple modalities. By developing core components for knowledge acquisition, incorporation and reasoning, I will be able to train high-performing and robust NLP systems that can generalize better given fewer training examples. Such components will improve the interpretability of those otherwise black-box neural networks. This research has the potential to improve the applicability of NLP systems across the board, including translation, dialogue, and summarization. Beyond the immediate impact on academia and industry, the proposed research program will train a new generation of first-class experts in NLP, with expertise in adjacent research areas such as machine learning, computer vision, cognitive science, and AI. With the necessary skills and experience, these professionals will contribute to Canadian academia, global and local industry.