CAREER: Efficient Learning of Personalized Strategies

在线零售商经常提供量身定制的产品或电影推荐。但是，由数据和统计驱动的自动化个性化的力量可能会大得多：想象一下，如果所有的孩子都有一个个性化的、自我改进的辅导系统作为他们教育的一部分，对减贫的影响会有多大。要实现这一愿景，需要个性化系统，既要考虑推荐项目的即时影响（例如，学习者是否会立即从视频讲座中学习），也要考虑其长期影响。例如，一个推荐的项目或干预可能会导致用户改变他/她的偏好、知识状态，或者揭示用户以前不知道的信息。这就需要创建个性化策略的方法：关于在什么情况下做出什么决定（是否显示或显示哪个广告，提供哪些教学活动）以最大化长期结果的适应性规则。本研究涉及开发新的数据驱动的机器学习方法，为相关个体构建这种个性化策略，并使用它们来提高在线数学教育系统的有效性。该项目将个性化策略创建框架为不确定性研究下的顺序决策。尽管在不确定性下的顺序决策方面已经取得了许多进展，但现有的方法主要集中在其他应用领域，如机器人，并且未能考虑或利用与人交互时出现的一些特殊功能。其中包括对人进行精确的模拟是困难的，但通常可以获得先前的数据，并且个体通常是相关的。该项目提供了挖掘现有数据集的算法，以创建和精确绑定新的高质量策略的预期性能，并在一系列类似的顺序决策任务中进行在线政策学习。

Online retailers frequently provide tailored product or movie recommendations. But the power of automated personalization, driven by data and statistics, could be far greater: imagine the impact on poverty reduction if all children had a personalized, self-improving tutoring system as part of their education. To realize this vision requires personalization systems that reason about both the immediate impact of a recommended item (e.g. will a learner immediately learn from a video lecture) as well as its longer term impact. For example, a recommended item or intervention may cause a user to change his/her preferences, state of knowledge, or reveal information about the user that was previously unknown. This requires methods for creating personalized strategies: adaptive rules about what decisions to make (whether or which ad to show, which pedagogical activity to provide) in which circumstances to maximize for long term outcomes. This research involves developing new data-driven, machine learning approaches to construct such personalized strategies for related individuals, and using them towards improving the effectiveness of online mathematics educational systems. The project frames personalized strategy creation as sequential decision making under uncertainty research. Though there have been many advances in sequential decision making under uncertainty, existing approaches have focused primarily on other application areas, like robotics, and fail to account or leverage for some of the special features that arise when interacting with people. These include that accurate simulation of people is difficult but prior data is often available, and that individuals are often related. This project contributes algorithms for mining existing datasets to create and precisely bound the expected performance of new high-quality strategies and for online policy learning across a series of similar sequential decision making tasks.