A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy.
A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy.
复制标题
DOI：
10.1088/1361-6560/ac09a2
复制
发表时间：
2021-06-23
影响因子：
3.5
通讯作者：
Jia X
中科院分区：
文献类型：
--
作者：
Shen C;Chen L;Jia X
我们之前已经提出了一个智能自动治疗计划(IATP)框架，它构建了一个虚拟治疗计划员网络(VTPN)来运行治疗计划系统(TPS)，以生成高质量的放射治疗(RT)治疗计划。虽然IATP在自动化RT治疗计划方面的潜力已经被证明，但由于网络规模几乎与治疗计划参数(TPP)的数量呈线性增长而导致的可扩展性差是一个瓶颈，阻碍了其在复杂但临床相关的治疗计划问题中的应用。训练好的网络的决策行为很难理解。在人类计划者决策过程的激励下，本研究提出了一个层次化的IATP框架。分层VTPN(HieVTPN)由三个网络组成，即结构网、参数网和动作网。在与TPS交互时，网络在每个步骤中按顺序分别决定要调整的结构、选择的结构调整的TPP和参数的具体调整方式。我们开发了一种端到端的分层深度强化学习(DRL)方案来同时训练这三个网络。然后，我们评估了所提出的框架在前列腺癌调强放疗(IMRT)和立体定向体部放疗(SBRT)治疗规划问题中的有效性。我们通过比较VTPN制定的并行架构的计划，以及在2016年美国医学剂量学家协会(AAMD)/放射外科学会(RSS)计划研究中提交竞争的人类计划，对我们方法的性能进行了基准比较。我们分析了网络规模相对于TPP数量的可扩展性。为了理解训练后的HieVTPN决策行为的理论基础，还进行了数值实验。使用10个训练病例和5个验证案例成功地训练了用于前列腺IMRT和SBRT的HieVTPN。对于IMRT，HieVTPN能够为59例未包括在培训过程中的测试患者生成高质量的计划，平均计划得分为8.62(±0.83)，最高得分为9。评分与VTPN相当，为8.45(±0.48)分。对于SBRT计划，HieVTPN在5个测试患者病例上的平均计划得分为139.07，而在美国儿科学会/随机对照试验计划研究中总结的人类计划的平均得分为132.21。与网络规模随TPP个数呈线性关系的VTPN不同，HieVTPN的网络规模几乎与TPP个数无关。还观察到，HieVTPN的决策行为是可以理解的，并与人类经验大体一致。层次化的IATP框架具有可扩展性和可解释性，在处理涉及大量TPP的治疗计划问题方面比以前的框架更有利。
We have previously proposed an intelligent automatic treatment planning (IATP) framework that builds a virtual treatment planner network (VTPN) to operate a treatment planning system (TPS) to generate high-quality radiation therapy (RT) treatment plans. While the potential of IATP in automating RT treatment planning has been demonstrated, its poor scalability caused by an almost linear growth of network size with the number of treatment planning parameters (TPPs) is a bottleneck, preventing its application in complicate, but clinically relevant treatment planning problems. The decision-making behavior of the trained network is hard to understand. Motivated by the decision-making process of a human planner, this study proposes a hierarchical IATP framework. The hierarchical VTPN (HieVTPN) consists of three networks, i.e. Structure-Net, Parameter-Net, and Action-Net. When interacting with a TPS, the networks are employed in a sequential order in each step to decide the structure to adjust, the TPP to adjust for the selected structure, and the specific adjustment manner for the parameter, respectively. We developed an end-to-end hierarchical deep reinforcement learning (DRL) scheme to simultaneously train the three networks. We then evaluated the effectiveness of the proposed framework in the treatment planning problems for prostate cancer intensity modulated RT (IMRT) and stereotactic body RT (SBRT). We benchmarked the performance of our approach by comparing plans made by VTPN of a parallel architecture, and the human plans submitted for competition in the 2016 American Association of Medical Dosimetrist (AAMD)/Radiosurgery Society (RSS) Plan Study. We analyzed scalability of the network size with respect to the number of TPPs. Numerical experiments were also performed to understand the rationale of the decision-making behaviors of the trained HieVTPN. Both HieVTPNs for prostate IMRT and SBRT were trained successfully using 10 training patient cases and 5 validation cases. For IMRT, HieVTPN was able to generate high-quality plans for 59 testing patient cases that were not included in training process, achieving an average plan score of 8.62 (±0.83), with 9 being the maximal score. The score was comparable to that of the VTPN, 8.45 (±0.48). For SBRT planning, HieVTPN achieved an average plan score of 139.07 on five testing patient cases compared to the score of 132.21 averaged over the human plans summited for competition in AAMD/RSS plan study. Different from VTPN with network size linearly scaling with the number of TPPs, the network size of HieVTPN is almost independent of the number of TPPs. It was also observed that the decision-making behaviors of HieVTPN were understandable and generally agreed with the human experience. With the scalability and explainability, the hierarchical IATP framework is more favorable than the previous framework in terms of handling treatment planning problems involving a large number of TPPs.
登录
查看更多内容
影响因子：
3.8
作者：
Shen C;Nguyen D;Chen L;Gonzalez Y;McBeth R;Qin N;Jiang SB;Jia X
通讯作者：
Jia X
影响因子：
3.5
作者：
Li, Xinyi;Zhang, Jiahan;Wang, Chunhao
通讯作者：
Wang, Chunhao
影响因子：
3.8
作者：
Wang, Huan;Dong, Peng;Xing, Lei
通讯作者：
Xing, Lei
影响因子：
3.8
作者：
Holdsworth, Clay;Kim, Minsun;Phillips, Mark
通讯作者：
Phillips, Mark
影响因子：
3.8
作者：
Holdsworth, Clay;Kim, Minsun;Phillips, Mark H.
通讯作者：
Phillips, Mark H.