CAREER: New Frontiers in Sequential Decision Making with a View Towards Mobile Health Applications

这个NSF CAREER项目使用机器学习来推进通过移动设备提供的健康干预科学。在全球移动设备广泛使用的推动下，移动医疗（mhealth）领域正在迅速发展。移动健康研究人员正在使用移动设备来诊断、管理和治疗各种健康状况，包括但绝不限于酗酒、抑郁、肥胖和吸烟。该项目创建了新的学习算法，学习个性化移动医疗干预措施，以满足个人用户不断变化的需求。它还开发了利用用户之间的社交网络进行干预的技术，以便将整个网络引向积极的健康结果。该项目将为本科生研究人员提供机会，研究由社会相关问题驱动的前沿数据科学。它将开发新的研究生水平的顺序决策课程。包括针对健康领域的模拟、学习算法的实现和提供干预措施的移动应用程序在内的软件将公开发布。在新成立的密歇根数据科学研究所（MIDAS）的支持下，研究结果将被传达给公众，并征求公众的反馈意见。密歇根大学的NextProf科学研讨会和教师多样性联盟项目将鼓励代表性不足的少数族裔的参与。该项目涉及解决三个技术难题。首先，现有的顺序决策算法，如上下文强盗算法，需要重新设计，使其适合在移动医疗中采用。该项目将开发演员-评论家上下文强盗算法，将策略的表示与奖励函数的表示分开。具有可解释的低维策略空间对于领域科学家学习策略的可解释性至关重要。其次，大多数关于强盗问题和强化学习的文献都假设了平稳性，这在移动医疗中是一个不合理的假设。利用无悔学习领域的最新进展，该项目将开发可以处理非平稳性的学习算法。第三，网络干预科学尚处于起步阶段。该项目将通过综合高维统计学习和复杂网络控制方面的最新进展，设计学习算法，干预时间进化网络中的少量节点，以实现预期的长期目标，从而为其发展提供催化剂。

This NSF CAREER project uses machine learning to advance the science of health interventions delivered via mobile devices. The field of mobile health (mhealth) is advancing rapidly spurred by the widespread use of mobile devices across the world. Mhealth researchers are using mobile devices to diagnose, manage, and treat a variety of health conditions including, but by no means limited to, alcohol abuse, depression, obesity, and smoking. The project creates new learning algorithms that learn to personalize mhealth interventions to the changing needs of individual users. It also develops techniques that use the social network among users to intervene so that the entire network is steered towards positive health outcomes. The project will give undergraduate researchers the opportunity to work on cutting edge data science driven by socially relevant problems. It will develop new graduate level courses on sequential decision making. Software including simulations specific to health domains, implementations of learning algorithms, and mobile apps that deliver interventions will be publicly released. Research results will be communicated to, and feedback solicited from, the general public under the auspices of the newly formed Michigan Institute for Data Science (MIDAS). Involvement of under-represented minorities will be encouraged via the NextProf Science workshop and Faculty Allies for Diversity program at the University of Michigan.The project involves solving three technical challenges. First, existing algorithms for sequential decision making, such as contextual bandit algorithms, need to be redesigned to make them suitable for adoption in mhealth. The project will develop actor-critic contextual bandit algorithms that separate the representation of the policy from the representation of the reward function. Having an interpretable, low dimensional policy space is crucial for interpretability of the learned policy by the domain scientist. Second, most of the literature on bandit problems and reinforcement learning assumes stationarity, an unreasonable assumption in mhealth. Using recent advances in the field of no-regret learning, the project will develop learning algorithms that can deal with non-stationarity. Third, the science of network interventions is in its infancy. The project will serve a catalyst for its development by synthesizing recent advances in high dimensional statistical learning and control of complex networks to design learning algorithms that intervene at a small number of nodes in a time evolving network to achieve a desired long term objective.