CAREER: Efficient Learning of Personalized Strategies

在线零售商经常提供量身定做的产品或电影推荐。但是，由数据和统计数据驱动的自动化个性化的力量可能要大得多：想象一下，如果所有孩子都有一个个性化的、自我改进的辅导系统作为他们教育的一部分，对减贫会产生什么影响。为了实现这一愿景，需要个性化系统来推断推荐项目的直接影响(例如，学习者是否会立即从视频讲座中学习)以及其长期影响。例如，推荐的项目或干预可能导致用户改变他/她的偏好、知识状态或揭示关于用户的先前未知的信息。这需要创建个性化策略的方法：关于做出什么决定(是否或显示哪个广告，提供什么教学活动)的适应性规则，在哪些情况下为长期结果最大化。这项研究涉及开发新的数据驱动的机器学习方法来为相关个人构建这样的个性化策略，并使用它们来提高在线数学教育系统的有效性。该项目将个性化战略创建框架为不确定性研究下的序贯决策。尽管在不确定情况下的顺序决策方面取得了许多进展，但现有的方法主要集中在其他应用领域，如机器人技术，没有考虑或利用与人互动时出现的一些特殊功能。这些问题包括，很难对人进行准确的模拟，但之前的数据往往是可用的，而且个人往往是相关的。该项目贡献了挖掘现有数据集的算法，以创建并精确限制新的高质量策略的预期性能，以及跨一系列类似的顺序决策任务的在线策略学习。

Online retailers frequently provide tailored product or movie recommendations. But the power of automated personalization, driven by data and statistics, could be far greater: imagine the impact on poverty reduction if all children had a personalized, self-improving tutoring system as part of their education. To realize this vision requires personalization systems that reason about both the immediate impact of a recommended item (e.g. will a learner immediately learn from a video lecture) as well as its longer term impact. For example, a recommended item or intervention may cause a user to change his/her preferences, state of knowledge, or reveal information about the user that was previously unknown. This requires methods for creating personalized strategies: adaptive rules about what decisions to make (whether or which ad to show, which pedagogical activity to provide) in which circumstances to maximize for long term outcomes. This research involves developing new data-driven, machine learning approaches to construct such personalized strategies for related individuals, and using them towards improving the effectiveness of online mathematics educational systems. The project frames personalized strategy creation as sequential decision making under uncertainty research. Though there have been many advances in sequential decision making under uncertainty, existing approaches have focused primarily on other application areas, like robotics, and fail to account or leverage for some of the special features that arise when interacting with people. These include that accurate simulation of people is difficult but prior data is often available, and that individuals are often related. This project contributes algorithms for mining existing datasets to create and precisely bound the expected performance of new high-quality strategies and for online policy learning across a series of similar sequential decision making tasks.