Learning from Instruction and Experience

这是一个为期3年的连续奖项的第一年。这项研究的目的是拓宽人类和机器学习者之间目前狭窄的“信息管道”；正在设计的算法还允许人类就手头的任务提供广泛适用的建议，而不是像通常所做的那样，仅仅向学习者提供标记的训练样本。人类的建议者不必理解机器学习器的内部表示和算法，所提供的建议也不必完全正确或完全明确。正在开发的算法使用连接主义强化学习技术来(缓慢地)从学习者环境提供的反馈中学习。建议者观察机器学习者的决策，偶尔会提出建议，用简单的编程语言表达为指令。基于基于知识的神经网络领域的技术，建议被直接插入代理的知识库中。重要的是，学习者随后可以通过基于进一步经验的神经训练和来自人类顾问的额外建议来提炼吸收的建议。这项工作承诺简化人类和机器学习者之间的交互，并可能产生灵活、适应性强的软件，通过该项目灵活的建议提供框架来调整自身以适应特定用户。

This is the 1st-year of a 3-year continuing award. The objective of this research is to widen the currently narrow `information pipeline` between humans and machine learners; instead of solely providing labeled training examples to the learner as is typically done, the algorithms being designed also allow humans to provide broadly applicable advice regarding the task at hand. The human advice-giver does not have to understand the machine learner's internal representations and algorithms, nor does the provided advice have to be completely correct or fully explicit. The algorithms being developed use techniques from connectionist reinforcement learning to (slowly) learn from the feedback provided by the learner's environment. The advice-giver observes the machine learner's decision making and occasionally makes suggestions, expressed as instructions in a simple programming language. Based on techniques from the field of knowledge-based neural networks, the advice is inserted directly into the agent's knowledge base. Importantly, the learner can subsequently refine the assimilated advice both by neural training based on further experience and additional advice from the human advisor. This work promises to simplify the interaction between humans and machine learners, and may lead to flexible, adaptive software that tunes itself to specific users via this project's flexible advice-giving framework.