EAGER: Rule Induction Games to Explore Differences between Human and Machine Intelligence

这个项目解决了人类和机器学习之间关系中一个以前未探索过的问题。 许多挑战人类智力的问题（国际象棋、围棋）都已经屈服于现代计算机算法。然而，一些对人类甚至动物来说很容易的任务--比如灵活的运动和对周围环境的快速而强大的视觉理解--仍然处于人工智能研究的前沿。计算机计算无误。然而，例如，相当多的人谁知道奇数和偶数之间的区别会说，798是奇数，也许是因为它的三分之二的数字是奇数。计算机学习的方式和人类学习的方式有什么根本区别吗？通过对游戏的严格研究，玩家必须通过试错来学习规则，他们能找到吗？该项目使用涉及规则学习的游戏来探索人类和机器学习之间的相似性和差异。它将寻求对人类学习的新见解，并可能提高对机器学习的理解。从长远来看，它的目标是更好地整合算法和人类来解决现实世界的问题;人类和计算机在相互补充的情况下能够最好地合作，该项目将寻求人类容易的规则和人类难以理解的规则之间的可概括的区别;特别关注的是发现机器的难度顺序正好相反的问题。找到这些逆转背后的原则将有助于分类问题。长期目标是混合系统，人类和机器学习集成，以实现医疗诊断，治疗计划等目标，该项目如果成功，将有助于严格定义一些对人类来说相对容易的学习问题如何以及为什么对机器来说更困难，反之亦然。通过对规则发现这一特定活动的关注，这项研究甚至可以为被称为“发现自然规律”的科学过程提供新的视角。 这个项目探索机器学习和人类学习之间的互补性，采用严格平衡的方法，使用向人类和计算机提出的“规则归纳”挑战。计算机将使用最先进的深度神经网络，并探索项目编码语言中可描述的规则的假设空间。心理学研究调查了关键问题，如跨规则的迁移学习，以及语言和命名在规则发现中的作用。 人类和机器“玩家”都是通过试错来学习规则的。规则编码语言、学习过程和评分系统确保了人类和机器学习者的对称性。 业绩衡量标准将包括折扣奖励和收敛到无差错的发挥。学习曲线将被用来衡量学习每一条规则的难度。实验条件将系统地变化，不仅包括要学习的规则，而且还包括参数，例如显示的不同形状的最小和最大数量，用户在尝试学习给定规则时可以使用的“板”的最大数量，以及玩家因其表现而获得奖励的激励/奖励结构。 这项研究将寻求识别成对的规则类，这样对人类来说更容易的类对计算机来说更困难，反之亦然。该项目将涉及使用各种机器学习方法的广泛实验，以及亚马逊Mechanical Turk的人类学习性能数据。“比较不同规则的可学习性为人类学习偏见提供了新的视角，可能有助于构建课程，并可能有助于确定哪些知识差距对人类解决问题最有害。 其目的是解释或解释这些异常的规则类与其他规则类的区别，其中人类和计算机的相对难度是相同的。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

This project tackles a previously unexplored problem in the relationship between human and machine learning. Many problems that challenge human intelligence (chess, Go) have yielded to modern computer algorithms. Yet some tasks that are easy for humans, or even animals — such as flexible locomotion and rapid and robust visual understanding of the surroundings are still at the cutting edge of artificial-intelligence research. Computers calculate without error. Yet, for example, quite a few people who know the difference between odd and even will say that 798 is odd, perhaps because two thirds of its digits are odd. Are there fundamental differences between the way computers learn and way humans learn? Can they be found with a rigorous study of games where the player must learn a rule by trial and error? This project uses games involving the learning of rules to explore similarities and differences between human and machine learning. It will seek new insights into human learning and may improve understanding of machine learning as well. Long-term, it aims to better integrate algorithms and humans for solving real-world problems; humans and computers work together best when they can complement each other, This project will seek generalizable distinctions between rules that are easy for humans and rules that are hard for humans; the special focus is to find problems where the order of difficulty is exactly reversed for machines. Finding the principles behind these reversals will help to triage problems. The long-term goal is hybrid systems, human and machine learning integrated to achieve goals such as medical diagnosis, treatment planning, etc. This project if successful will contribute to rigorously defining how and why some learning problems that seem relatively easier for humans are nonetheless more difficult for machines, and vice versa. With a focus on the specific activity of rule finding, this research may even shed new light on the scientific process, which has been characterized as “discovering the rules of nature.” This project explores complementarity between Machine Learning and Human Learning with a rigorously balanced approach, using a “rule induction” challenge that is presented to both humans and computers. Computers will use state-of-the art deep neural networks, and explore the hypothesis space of rules describable in the project’s coding language. The psychological research investigates crucial problems such as transfer learning across rules, and the role of language and naming in rule discovery. Both human and machine “players” learn the rules by trial and error. The rule encoding language, reinforcement-learning processes, and scoring systems ensure symmetry of human and machine learners. Performance measures will include discounted reward and convergence to error-free play. Learning curves will be used to measure the difficulty of learning each rule. Experimental conditions will be systematically varied, including not only the rule to be learned, but also parameters such as the minimum and maximum number of different shapes displayed, the maximum number of “boards” that a user may use in attempting to learn a given rule, and the incentive/reward structure by which players earn rewards for their performance. The research will seek identify pairs of classes of rules such that the class that is easier for humans is more difficult for computers, and vice versa. The project will involve extensive experiments using diverse machine-learning approaches, as well as Amazon Mechanical Turk for data on human learning performance."Comparing the learnability of different rules sheds new light on human learning biases, may prove useful for structuring curricula, and may help identify which gaps in knowledge are most detrimental to human problem solving. The goal is to interpret or explain what distinguishes these anomalous pairs of rule classes from others where the relative degree of difficulty is the same for humans and computers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.