Deep Learning for Time-Inconsistent Dynamic Optimization

这项拟议的研究旨在解决所谓的时间不一致(TI)动态优化问题，该问题解决了在存在不一致且经常相互冲突的人类行为的情况下进行决策的问题，例如，戒烟的长期健康益处与对尼古丁渴望的即时愉悦。从消费者行为到社会福利政策，解决TI动态优化可以产生深远的影响。技术创新框架下的决策完全不同于理性行为假设下的标准最优化决策和经济理论决策。针对TI动态优化的研究成果寥寥无几。由于求解高维非线性偏微分方程组和正倒向随机微分方程组的要求，主要的瓶颈是计算。该项目旨在通过将数据科学中的深度强化学习(DRL)与凸分析、对偶随机控制等先进的数学理论相结合，发展解决TI动态优化的基础理论和新方法。一个例子是资产配置，许多金融机构使用一期均值方差(MV)模型，该模型使用起来简单，但有许多缺点。对于随机资产价格过程，多周期或连续时间模型更符合实际，更符合经济的动态性质，但具有一定的时间依赖性，求解难度较大。该项目的发现有助于解决持续时间MV问题，这些问题将改善金融资产负债管理和业绩，进而对社会繁荣和个人福祉产生重大影响。总之，TI动态优化和DRL计算的进展可以极大地帮助行业和政府机构改进决策，并基于项目中开发的DRL解算器设计更高效、更强大的计算软件来解决真实的TI问题。

The proposed research is to solve a so called time-inconsistent (TI) dynamic optimization problem that addresses decision making in the presence of inconsistent and often conflicting human behaviour, e.g., long term health benefit of stopping smoking vs instant pleasure of nicotine cravings. Solving TI dynamic optimization can have far-reaching impact from consumer behaviour to social welfare policy. The decision making under the TI framework is completely different from ones in standard optimization and economic theory under the rational behaviour assumption. The results for TI dynamic optimization are few and far between. The main bottleneck is computation due to the requirement of solving the system of high dimensional nonlinear partial differential equations and forward-backward stochastic differential equations. The project is to develop the fundamental theory and novel methodology to solve TI dynamic optimization by integrating the deep reinforcement learning (DRL)} from data science with advanced mathematical theories such as convex analysis, dual stochastic control, etc. The breakthrough in solving TI dynamic optimization can make great impact in applications. One example is asset allocation, many financial institutions use one-period mean variance (MV) model, which is simple to use but has many drawbacks. A multi-period or continuous time model is more realistic for stochastic asset price processes and fits better the dynamic nature of the economy, but is TI and difficult to solve. The findings of the project can help solve continuous time MV problems that would improve financial asset liability management and performance, which in turn would have great impact on societal prosperity and individual well-being. In short, progress in TI dynamic optimization and DRL computation can greatly help industry and government agencies to improve decision making and design more efficient and powerful computational software for real-world TI problems, based on the DRL solver developed in the project.