Study on Emergent Functional Evolution and Heuristic Knowledge Acquisition by Neural Structural Development

本研究通过分析双反馈神经元（DFNU）的神经放电动力学来阐明神经放电通信和功能形成的生理微观机制。分析的一个重要重点是使用DFNU的简单配方，可以提供定量的分析结果。将Hodgkin-Huxley模型的动力学与DFNU模型的动力学进行了比较，结果表明，在一定条件下，DFNU模型的动力学在生理上也是合理的。结果表明，高频放电是相对合适的神经信息载体，由于可靠性和鲁棒性噪声输入。为了实现这种可靠的尖峰通信，我们通过使用具有适当幅度的额外噪声输入来改善DFNU的性能。仿真研究表明，存在最佳振幅区域，使DFNU具有类似于随机共振现象的噪声增强的可靠通信能力，除了微观分析外，我们进一步进行实验分析，从宏观脑科学的角度探讨了突现功能进化的关键机制。为此，提出了一种用于控制非完整系统的分阶段强化学习算法。该算法的关键元素是一个新的位置-方向空间上定义的成形函数。一旦达到目标，就开始构造成形函数，并约束探索策略。该方法被应用到一个2连杆平面欠驱动机械手的定位任务。这种机械手只有一个被动关节，难以控制。仿真结果表明，该成形函数在学习速度和最优策略方面都是有效的。

In this research project, we analyze neural spike dynamics of a double feedback neural unit (DFNU) to clarify physiological microscopic mechanism of neural spike communication and functional formation. An essential emphasis of the analysis is on use of the DFNU's simple formulations that can provide quantitative analytic results. Comparing dynamics of Hodgkin-Huxley model to that of the DFNU, it is shown that dynamics of the DFNU is also physiologically plausible under a condition. The results suggest that high-frequency firings are relatively appropriate for a neural informational carrier due to the reliability and robustness to noisy inputs. To realize such reliable spike communication, we improved the DFNU's performance by using extra noisy inputs with appropriate amplitudes. Simulation studies show that there is optimal region of the amplitude that makes the DFNU possess the noise-enhanced reliable communication ability as similar to stochastic resonance phenomena.In addition to the microscopic analysis, we further conduct experimental analysis to investigate a key mechanism of emergent functional evolution from a viewpoint of macroscopic brain sciences. A phased reinforcement learning algorithm for controlling nonholonomic systems is proposed for this purpose. The key element of the proposed algorithm is a shaping function defined on a novel position-direction space. The shaping function is started to be constructed once the goal is reached and constrains the exploration strategy. The proposed method is applied to the positioning tasks of a 2-link planer underactuated manipulator. This manipulator has one passive joint and is difficult to control. As the result, the efficiency of the proposed shaping function was confirmed in learning speed and best policy.