CAREER: Reading To Learn: Language-Guided Machine Learning

几个世纪以来，人类使用语言是为了相互交流，并将知识传递给后代。我们通过"做“事情来接收来自世界的反馈（例如，当我们把手指放在火里时感到疼痛）以及”阅读"世界是如何运作的（例如，维基百科可能会说“火有可能通过燃烧引起疼痛和身体伤害”）。现代人工智能（AI）系统主要通过前一种方法学习新技能，使用试错机制，需要将自己的预测与人类指定的答案或判断进行比较。虽然这种方法已经用于自动化各种任务，但它需要大量的数据和计算资源，并且由于涉及的风险较低，因此仅限于适合试错学习的任务域。该项目将为语言引导的机器学习的新范式开发技术，使人工智能系统能够通过阅读自然语言中的相关文本（如书籍，手册和网页）来获得新的知识和技能。这将产生强大的AI模型，需要更少的人力来训练，同时允许更好的用户个性化。目前有效的机器学习方法，如域自适应，少次学习，持续学习和强化学习，只能在模型开发人员预先指定的特定于任务的符号或数学表示（如类ID或层次结构，动态模型，奖励函数）上操作，并且不能利用语言知识提供相同的信息。这个CAREER项目将开发可以“阅读”的模型，以从文本来源获取知识，并将其纳入不同范式的更好的学习过程。这包括监督分类任务以及顺序决策，其中代理在交互式环境中执行多个动作。可以通过阅读文本（从书籍、网页或人类反馈中）自动获取新知识和技能的模型将需要更少量的传统监督，更好地推广到看不见的场景，并大大减少模型开发中的人力。该项目将通过解决三个关键方向来实现这一目标：（1）通过开发一个提供语义类描述的新框架来实现语言指导的监督学习，（2）通过利用离线文本指导来提高强化学习中新领域的效率和泛化能力，以及（3）通过人机协作使用语言反馈来实现在线调整政策。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Humans have used language for centuries in order to communicate with each other and pass knowledge to successive generations. We learn through a combination of 'doing' things to receive feedback from the world (e.g. feeling pain when we put our finger in the fire) as well as 'reading' about how the world works (e.g. Wikipedia might say 'Fire has the potential to cause pain and physical damage through burning'). Modern artificial intelligence (AI) systems learn new skills predominantly through the former method, using a trial-and-error mechanism that requires comparing their own predictions against human-specified answers or judgements. While this approach has worked for automating a variety of tasks, it requires a large amount of data and computational resources, and is limited to task domains where trial-and-error learning is appropriate due to the low stakes involved. This project will develop techniques for a new paradigm of language-guided machine learning that will enable AI systems to acquire new knowledge and skills by reading relevant text in natural language such as books, manuals and webpages. This will result in robust AI models that require less human effort to train while allowing for better user personalization. Current approaches to efficient machine learning such as domain adaptation, few-shot learning, continual learning and reinforcement learning can only operate over task-specific symbolic or mathematical representations pre-specified by model developers (such as class IDs or hierarchies, dynamics models, reward functions) and do not leverage linguistic knowledge providing the same information. This CAREER project will develop models that can ‘read’ to acquire knowledge from textual sources and incorporate it into a better learning process for different paradigms. This includes supervised classification tasks as well as sequential decision-making where an agent executes several actions in an interactive environment. Models that can automatically acquire new knowledge and skills by reading text (from books, webpages, or human feedback) will require smaller amounts of traditional supervision, generalize better to unseen scenarios, and substantially reduce human effort in model development. The project will achieve this goal by tackling three key directions: (1) enabling language-guided supervised learning by developing a new framework for providing semantic class descriptions, (2) improving efficiency and generalization to new domains in reinforcement learning by leveraging offline textual guidance, and (3) enabling online adaptation of policies using linguistic feedback through human-machine collaboration. These thrusts will open new research directions for machine learning with language guidance and enable better real-world human-machine collaboration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.