Reinforced Imitative Graph Representation Learning for Mobile User Profiling: An Adversarial Training Perspective
Reinforced Imitative Graph Representation Learning for Mobile User Profiling: An Adversarial Training Perspective
复制标题
DOI：
10.1609/aaai.v35i5.16567
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Dongjie Wang;Pengyang Wang;Kunpeng Liu;Yuanchun Zhou;C. Hughes;Yanjie Fu
中科院分区：
文献类型：
--
作者：
Dongjie Wang;Pengyang Wang;Kunpeng Liu;Yuanchun Zhou;C. Hughes;Yanjie Fu
在本文中，我们研究了移动用户分析问题，这是人类移动建模管道中量化用户特征的关键组成部分。人的移动性是一个依赖于用户动态兴趣的连续决策过程。基于准确的用户资料，该预测模型可以完美再现用户的移动轨迹。反过来，一旦预测模型能够模仿用户的移动模式，学习到的用户轮廓也是最优的。这种直觉促使我们通过利用强化学习提出一种基于模仿的移动用户分析框架，其中训练代理精确模仿用户的移动模式以获得最佳用户配置文件。具体而言，该框架包括两个模块：(1)表示模块，该模块结合用户概况和时空上下文实时生成状态；(2)模仿模块，其中Deep Q-network （DQN）根据表示模块产生的状态来模仿用户的行为（动作）。然而，要有效地运行该框架存在两个挑战。首先，DQN中的epsilon-greedy策略通过随机选择具有epsilon概率的动作来利用探索-利用的权衡。这种随机性反馈给表示模块，导致学习到的用户配置文件不稳定。为了解决这个问题，我们提出了一种对抗训练策略来保证表示模块的鲁棒性。其次，表示模块以增量方式更新用户配置文件，需要集成用户配置文件的时间效应。受长短期记忆（LSTM）的启发，我们引入了一种门控机制，将新老用户特征合并到用户配置文件中。在实验中，我们在真实世界的数据集上评估了我们提出的框架。大量的实验结果验证了我们的方法与基线算法相比的优越性。
In this paper, we study the problem of mobile user profiling, which is a critical component for quantifying users' characteristics in the human mobility modeling pipeline. Human mobility is a sequential decision-making process dependent on the users' dynamic interests. With accurate user profiles, the predictive model can perfectly reproduce users' mobility trajectories. In the reverse direction, once the predictive model can imitate users' mobility patterns, the learned user profiles are also optimal. Such intuition motivates us to propose an imitation-based mobile user profiling framework by exploiting reinforcement learning, in which the agent is trained to precisely imitate users' mobility patterns for optimal user profiles. Specifically, the proposed framework includes two modules: (1) representation module, that produces state combining user profiles and spatio-temporal context in real-time; (2) imitation module, where Deep Q-network (DQN) imitates the user behavior (action) based on the state that is produced by the representation module. However, there are two challenges in running the framework effectively. First, epsilon-greedy strategy in DQN makes use of the exploration-exploitation trade-off by randomly pick actions with the epsilon probability. Such randomness feeds back to the representation module, causing the learned user profiles unstable. To solve the problem, we propose an adversarial training strategy to guarantee the robustness of the representation module. Second, the representation module updates users' profiles in an incremental manner, requiring integrating the temporal effects of user profiles. Inspired by Long-short Term Memory (LSTM), we introduce a gated mechanism to incorporate new and old user characteristics into the user profile. In the experiment, we evaluate our proposed framework on real-world datasets. The extensive experimental results validate the superiority of our method comparing to baseline algorithms.