Adaptive Scanning in Ptychography through Deep Reinforcement Learning
Adaptive Scanning in Ptychography through Deep Reinforcement Learning
复制标题
DOI：
10.1017/s1431927621003238
复制
发表时间：
2021-07
影响因子：
2.8
通讯作者：
M. Schloz;Johannes Müller;T. Pekin;W. Van den Broek;C. Koch
中科院分区：
文献类型：
--
作者：
M. Schloz;Johannes Müller;T. Pekin;W. Van den Broek;C. Koch
新的硬件和重建算法的发展使得平面摄影成为一种成熟的电子显微镜技术。目前进一步改进该技术的研究是由研究厚样品以及低电子剂量测量的愿望驱动的。对于后者，研究试图放宽通过连续重叠探针获得信息冗余所需的严格扫描约束。一种方法是优化扫描模式。从栅格向通常更有益的图案的偏离被认为可以改善重建质量，特别是在降低剂量[1]时。然而，由于一般优化的扫描方式与所研究样品的结构完全无关，因此可以实现的重建质量提高显然是有限的。在这里，我们提出了一种自适应扫描算法，该算法基于从显微镜获得的衍射数据中接收到的结构信息来优化扫描模式。我们的算法基于深度学习[2]和强化学习（RL）[3]或深度RL的结合，这意味着我们用深度神经网络对所描述的预测过程进行建模，并使用RL在先前获得的神经学数据上进行训练。采用递归神经网络（RNN）从初始子序列出发，预测扫描位置的子序列。输入信息由当前扫描的子序列rp的处理过的二维坐标和处理过的内容信息zp的组合xp给出，zp是部分型图重建vp的压缩表示。此外，所有来自前一个预测步骤的信息，由隐藏状态hp表示，与输入信息xp结合，并由一个门控循环单元（GRU）单元[4]映射到下一个隐藏状态
The development of new hardware and reconstruction algorithms has led to ptychography becoming a mature electron microscopy technique. The current research to further improve this technique is driven by the desire to investigate thick samples as well as to measure with a low electron dose. For the latter, researches have tried to relax the strict scanning constraint, which is required for obtaining information redundancy through consecutively overlapping probes. One approach is the optimization of the scanning pattern. A deviation from a raster grid towards a generally more beneficial pattern is supposed to improve the reconstruction quality especially at a reduced dose [1]. However, since a generally optimized scanning pattern is completely unrelated to the structure of the investigated specimen, there is obviously a limit of the achievable reconstruction quality improvement. Here, we present an adaptive scanning algorithm that optimizes the scanning pattern based on structural information it receives from diffraction data that is acquired by the microscope. Our algorithm is based on a combination of deep learning [2] and reinforcement learning (RL) [3], or deep RL, which means that we model the described prediction process with deep neural networks and train it on previously acquired ptychographic data with RL. A recurrent neural network (RNN) is used to predict sub-sequences of scanning positions after starting from an initial sub-sequence. Input information is given by a combination X p of processed 2D coordinates of a currently scanned sub-sequence R p and processed content information z p that is a compressed representation of the partial ptychographic reconstruction V p . In addition, all the information from the previous prediction steps, represented by the hidden state H p , is combined with the input information X p and mapped by a gated recurrent unit (GRU) cell [4] to the next hidden state