Collaborative Research: Intrinsically Motivated Learning in Artificial Agents

协作研究：人工智能体的内在动机学习

• 批准号: 0432027
• 负责人: Satinder Baveja
• 金额: $ 15万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-09-01 至 2007-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0432027&HistoricalAwards=false
• 关键词: Collaborative; Research; Intrinsically; Motivated; Learning

Collaborative Research: Intrinsically Motivated Learning in Artificial AgentsProject SummaryHumans are unendingly curious; we spontaneously explore and manipulate our surroundings to see what we can make them do; we obtain enjoyment from making discoveries and for making things happen. We often engage in these activities for their own sakes rather than as steps toward solving practical problems. Psychologists call these intrinsically motivated behaviors because rewards are intrinsic in these activities instead of being due to the satisfaction of more primary biological needs. But what we learn during intrinsically motivated behavior is essential for our development as competent autonomous entities able to efficiently solve a wide range of practical problems as they arise. This project's objective is to develop a computational model of intrinsically motivated learning that will allow artificial agents to construct and extend hierarchies of reusable skills that are needed for competent autonomy. This project builds on existing research in machine learning, recent advances in the neuroscience of brain reward systems, and classical and contemporary psychological theories of motivation. At the core of the model are recent theoretical and algorithmic advances in computational reinforcement learning, specifically, new concepts related to skills and new learning algorithms for learning with skill hierarchies. The project develops a mathematical framework, implements the model in a series of simulated agents, and demonstrates the advances this will make possible in a series of increasingly complex environments.Intellectual Merit-Machine learning methods have become much more powerful in recent years. Despite these advances and their utility, today's learning algorithms fall far short of the possibilities for machine learning. They are typically applied to single, isolated problems for each of which they have to be hand-tuned and for which training data sets have to be carefully prepared. They do not have the generative capacity required to significantly extend their abilities beyond initially built-in representations. They do not address many of the reasons that learning is so useful in allowing animals to cope with new problems as they arise over extended periods of time. Success in this project will provide a fundamental advance in machine learning and move the field in a new direction. Although computational study related to intrinsic motivation is not entirely new, it is currently underdeveloped and does not take advantage of the highly relevant recent advances in the field of computational reinforcement learning and in the neuroscience of brain reward and motivation systems. Furthermore, computational studies do not take advantage of psychological theories of play, curiosity, surprise, and other factors involved in intrinsically motivated learning. This project addresses these shortcoming by taking an explicitly interdisciplinary approach.Broader Impacts-The new methods promise to improve our ability to control the behavior of complex systems in ways that will benefit society. Machine learning algorithms have been instrumental in a wide variety of applications in such areas as bioinformatics, manufacturing, communications, robotics, and security systems. It is important strategically, economically, and intellectually to increase the power of machine learning technologies as rapidly as possible. This project attempts to address some of these challenges. This project will strengthen interdisciplinary ties between the machine learning community of computer science and various disciplines devoted to the study of human cognitive development and education. The specific methods of concern in the proposed research have not yet been integrated. There has been verylittle cross-fertilization between the psychological study of intrinsic motivation and machine learning. The proposed research will remedy this situation, thereby helping to create an avenue of communication that can foster future developments in both fields. The project has the potential to contribute to our understanding of general principles underlying human cognitive development, with implications for education, where enhancing intrinsic motivation is a key factor.The educational component of the project focuses on graduate education through its training of graduate students. This includes the offering interdisciplinary graduate-level seminars at both U. of Massachusetts and U. of Michigan, to be taught by the PIs on the topic of intrinsically motivated learning. In its recruitment of graduate students, the project will take advantage of the role that U. Massachusetts plays as the lead institution in the NSF funded Northeast Alliance, which supports and mentors underrepresented minority students interested in academic careers in a science, mathematics, or engineering discipline. At U. of Michigan special effort will be made to recruit and involve undergraduates in student projects leading to summer projects funded by the Marian Sarah Parker Scholars Program, which targets female undergraduates and provides funds for summer research opportunities.

合作研究：人工智能体中的内在动机学习项目摘要人类有着无尽的好奇心;我们自发地探索和操纵我们的周围环境，看看我们能让他们做什么;我们从发现和使事情发生中获得乐趣。我们经常为了这些活动本身而从事这些活动，而不是作为解决实际问题的步骤。心理学家称这些行为为内在动机行为，因为奖励是这些活动中固有的，而不是由于满足更基本的生物需求。但是，我们在内在动机行为中所学到的东西，对于我们发展成为有能力的自主实体，能够有效地解决出现的各种实际问题至关重要。该项目的目标是开发一个内在动机学习的计算模型，该模型将允许人工代理构建和扩展胜任自主所需的可重用技能的层次结构。该项目建立在机器学习现有研究、大脑奖励系统神经科学的最新进展以及经典和当代动机心理学理论的基础上。该模型的核心是计算强化学习的最新理论和算法进展，特别是与技能相关的新概念和用于技能层次学习的新学习算法。该项目开发了一个数学框架，在一系列模拟代理中实现了该模型，并展示了在一系列日益复杂的环境中可能实现的进步。智能优点-机器学习方法近年来变得更加强大。尽管有这些进步和实用性，但今天的学习算法远远达不到机器学习的可能性。它们通常应用于单个的、孤立的问题，对于每个问题，它们都必须手动调整，并且必须仔细准备训练数据集。他们没有足够的生成能力来显著扩展他们的能力，超越最初内置的表征。他们没有解决许多原因，学习是如此有用，使动物能够科普新的问题，因为他们出现了很长一段时间。该项目的成功将为机器学习提供根本性的进步，并将该领域推向一个新的方向。虽然与内在动机相关的计算研究并不完全是新的，但它目前还不发达，并且没有利用计算强化学习领域和大脑奖励和动机系统神经科学领域的高度相关的最新进展。此外，计算研究并没有利用游戏，好奇心，惊喜和其他内在动机学习中涉及的因素的心理学理论。这个项目通过采取明确的跨学科方法来解决这些缺点。更广泛的影响-新方法有望提高我们控制复杂系统行为的能力，从而造福社会。机器学习算法在生物信息学、制造业、通信、机器人和安全系统等领域的各种应用中发挥了重要作用。在战略、经济和智力上，尽快提高机器学习技术的能力是非常重要的。本项目试图解决其中的一些挑战。该项目将加强计算机科学的机器学习社区与致力于人类认知发展和教育研究的各个学科之间的跨学科联系。拟议研究中所关注的具体方法尚未整合。内在动机的心理学研究和机器学习之间几乎没有交叉。拟议的研究将纠正这种情况，从而有助于建立一个沟通渠道，促进这两个领域的未来发展。该项目有可能有助于我们理解人类认知发展的一般原则，并对教育产生影响，其中增强内在动机是一个关键因素，该项目的教育部分侧重于通过培训研究生进行研究生教育。这包括提供跨学科的研究生水平的研讨会，在这两个美国。马萨诸塞州和美国密歇根州，将由PI教授关于内在动机学习的主题。在招收研究生的过程中，该项目将充分利用美国在这方面的作用。马萨诸塞州在NSF资助的东北联盟中担任领导机构，该联盟支持和指导对科学，数学或工程学科的学术生涯感兴趣的少数民族学生。在联合密歇根大学将做出特别努力，招募本科生并让他们参与学生项目，这些项目将导致由玛丽安萨拉帕克学者计划资助的夏季项目，该计划针对女本科生并为夏季研究机会提供资金。