A Cooperative Multiagent Reinforcement Learning Framework for Droplet Routing in Digital Microfluidic Biochips
A Cooperative Multiagent Reinforcement Learning Framework for Droplet Routing in Digital Microfluidic Biochips
复制标题
DOI：
10.1109/tcad.2022.3233019
复制
发表时间：
2023-09
影响因子：
2.9
通讯作者：
Chen Jiang;Rongquan Yang;Qi Xu;Hailong Yao;Tsung-Yi Ho;Bo Yuan
中科院分区：
文献类型：
--
作者：
Chen Jiang;Rongquan Yang;Qi Xu;Hailong Yao;Tsung-Yi Ho;Bo Yuan
数字微流控生物芯片（DMFB）在自动执行生化协议方面显示出巨大的优势，其通过操纵并行传输的离散纳/皮升液滴来实现高通量结果。然而，由于电极退化，液滴输送可能失败，导致不正确的流体操作。为了执行安全关键的生物协议，液滴输送的可靠性成为DMFB的最大关注点。以前的工作已经表明，可以使用基于强化学习（RL）的方法通过捕获电极的潜在健康状况并在线决策来学习可靠的运输策略。然而，先前的RL方法可能无法完成具有多个液滴的路由任务，因为不同代理之间缺乏合作（每个代理代表一个液滴）。为了解决这个问题并将RL方法扩展到许多液滴，本文提出了一种新的合作集中式学习和分布式执行多代理RL（MARL）框架，用于使用值分解网络（VDNs）在DMFB中进行液滴路由。此外，为了加快训练和决策过程以及将我们的方法应用于大型生物芯片，我们使用了部分观察空间，其中代理只能在以自身为中心的有限视场（FOV）中观察环境。与现有的方法相比，该方法在不同的DMFB上的成功率和平均完成时间方面表现出上级性能。我们还在大型生物芯片上验证了我们的方法（例如，$\mathbf {50\times 50}$DMFB），比现有技术方法（例如，十滴）。
Digital microfluidic biochips (DMFBs) have shown great advantages in automatically executing biochemical protocols through manipulating discrete nano/picoliter droplets which are transported in parallel to achieve high-throughput outcomes. However, because of electrode degradations, the droplet transportation may fail, causing incorrect fluidic operations. To perform safety-critical bio-protocols, the reliability of droplet transportation becomes an utmost concern for DMFBs. It has been shown by the previous works that a reliable transportation policy can be learned using reinforcement learning (RL)-based methods by capturing the underlying health conditions of electrodes and making online decisions. However, previous RL methods may fail to accomplish routing tasks with multiple droplets, because there is a lack of cooperation among different agents (each agent represents one droplet). To deal with this problem and scale RL methods to many droplets, this article proposes a new cooperative centralized learning and distributed execution multiagent RL (MARL) framework for droplet routing in DMFBs using value-decomposition networks (VDNs). Moreover, to speed up the training and decision process as well as apply our method in large biochips, we use a partial observation space where agents can only observe environment in a limited field of view (FOV) centered around themselves. Compared with the state-of-the-art approach, the superior performance of the proposed approach is demonstrated on different DMFBs in terms of success rate and average completion time. We also validate our method on large biochips (e.g., $\mathbf {50\times 50}$ DMFBs) with more droplets than state-of-the-art approach (e.g., ten droplets).