Deep Learning for Time-Inconsistent Dynamic Optimization

提出的研究是解决所谓的时间不一致（TI）动态优化问题，该问题解决了存在不一致且经常冲突的人类行为的决策问题，例如，戒烟的长期健康益处与尼古丁渴望的即时快感。解决TI动态优化可以产生深远的影响，从消费者行为到社会福利政策。TI框架下的决策与标准优化和理性行为假设下的经济理论中的决策完全不同。TI动态优化的结果很少。由于求解高维非线性偏微分方程组和正倒向随机微分方程组的要求，其主要瓶颈是计算量。该项目旨在通过将数据科学中的深度强化学习（DRL）与凸分析，对偶随机控制等先进数学理论相结合，开发解决TI动态优化的基础理论和新方法。解决TI动态优化的突破可以在应用中产生重大影响。其中一个例子是资产配置，很多金融机构使用的是单期均值方差（MV）模型，该模型使用简单但存在很多弊端。一个多周期或连续时间模型是更现实的随机资产价格过程，更好地适应经济的动态性质，但TI和难以解决。该项目的研究结果可以帮助解决持续时间MV问题，从而改善金融资产负债管理和绩效，进而对社会繁荣和个人福祉产生重大影响。简而言之，TI动态优化和DRL计算的进展可以极大地帮助行业和政府机构改善决策制定，并基于该项目开发的DRL求解器为现实世界的TI问题设计更高效、更强大的计算软件。

The proposed research is to solve a so called time-inconsistent (TI) dynamic optimization problem that addresses decision making in the presence of inconsistent and often conflicting human behaviour, e.g., long term health benefit of stopping smoking vs instant pleasure of nicotine cravings. Solving TI dynamic optimization can have far-reaching impact from consumer behaviour to social welfare policy. The decision making under the TI framework is completely different from ones in standard optimization and economic theory under the rational behaviour assumption. The results for TI dynamic optimization are few and far between. The main bottleneck is computation due to the requirement of solving the system of high dimensional nonlinear partial differential equations and forward-backward stochastic differential equations. The project is to develop the fundamental theory and novel methodology to solve TI dynamic optimization by integrating the deep reinforcement learning (DRL)} from data science with advanced mathematical theories such as convex analysis, dual stochastic control, etc. The breakthrough in solving TI dynamic optimization can make great impact in applications. One example is asset allocation, many financial institutions use one-period mean variance (MV) model, which is simple to use but has many drawbacks. A multi-period or continuous time model is more realistic for stochastic asset price processes and fits better the dynamic nature of the economy, but is TI and difficult to solve. The findings of the project can help solve continuous time MV problems that would improve financial asset liability management and performance, which in turn would have great impact on societal prosperity and individual well-being. In short, progress in TI dynamic optimization and DRL computation can greatly help industry and government agencies to improve decision making and design more efficient and powerful computational software for real-world TI problems, based on the DRL solver developed in the project.