Adaptive Power Management of IoT Systems by Reinforcement Learning

本年度的研究成果有两个方面：1)提出了能量采集无线传感器节点能量中性操作的多目标强化学习(MORL)方法；2)准备了博士论文。然而，使用这种方法不能优化EHWSN中不同任务之间的能量调度。因此，我研究了MORL方法来优化现代EHWSNs执行的多个任务的能量调度。由于以前的RL方法不能沿Pareto空间进行运行时权衡和/或需要太多的资源，我为EHWSNs开发了一个新的多目标RL框架。该框架可以动态地对多个目标进行优化和权衡。与直接的多目标RL方法相比，它消耗的资源要少得多，这使得它们适合于资源受限的EHWSN。我对EHWSN系统进行了改造，开发了一种多目标马尔可夫决策过程(MDP)，并提出了两种新的MORL算法，使EHWSN能够在更短的时间段内学习权衡策略，同时犯下更少的错误。我们关于这一新方法的论文目前正在审查中。我还合并了与我之前的研究论文不同的结果，以准备我的博士论文和答辩报告。我的研究(由这个KAKENHI支持)在研究界也得到了高度的热情，我被邀请担任与SenSys 2020(虚拟)共处的AIChallengeIoT 2020研讨会的宣传主席。

The research achievement for this year was two fold - 1)developing a Multi-Objective Reinforcement Learning (MORL) approach for Energy Neutral Operation (ENO) of Energy Harvesting Wireless Sensor Nodes (EHWSNs), and 2) preparing the PhD thesis.My previous research efforts concentrated on achieving ENO using a single reward function. However, one cannot optimize energy scheduling between various tasks in EHWSNs with this approach. So, I investigated into MORL methods to optimize energy scheduling over multiple tasks that modern EHWSNs execute.Since previous RL methods could not perform runtime tradeoffs along the Pareto-space and/or require prohibitively large amounts resources, I developed a novel multi-objective RL framework for EHWSNs. This framework can optimize over multiple objectives and tradeoff dynamically. It consumes much less resources compared to direct multi-objective RL methods making them suitable for resource constrained EHWSNs. I remodeled the EHWSN system, developed a multi-objective Markov Decision Process (MDP) and proposed two novel MORL algorithms that enables EHWSNs to learn tradeoff policies in shorter time periods while making lesser mistakes. Our paper on this novel method is currently under review.I also consolidated the different results from my previous research papers to prepare my PhD thesis and defense presentation. My research (supported by this KAKENHI) was also met with high enthusiasm in the research community and I was invited to be the publicity chair of AIChallenge IoT 2020 Workshop that was co-located (virtually) with SenSys 2020.