基于强化学习的仿生机器鱼节能研究
|
摘要
利用仿生机器鱼平台,模拟鱼类集群游动过程,通过实验方法探究机器鱼集群游动时的能量节省问题。具体地,将两条机器鱼按照一定的前后距离和左右距离固定在低湍流水洞中,并正对来流方向;控制前方(领航)机器鱼游动姿态的变化,使得后方(跟随)机器鱼处在变化的流场环境中。基于强化学习理论,建立跟随机器鱼在该流场环境下游动所消耗的能量值与其游动姿态之间的映射关系;利用该映射关系,获得跟随机器鱼能量消耗最小值对应的游动姿态信息,并将该信息反馈至机器鱼运动控制器中,调整机器鱼游动姿态,达到节省机器鱼能量的目标。
The authors studied energy saving in fish school utilizing a bio-inspired robotic fish platform.Specifically, two robotic fish were fixed in the flow tank against the current with constant left-right distance and front-back distance; the body posture of the front robotic fish was controlled to generate dynamic fluid environments, which would affect the rear robotic fish. Based on reinforcement learning, a relationship was established between energy cost and the body posture of the robotic fish. According to this relationship, the body posture could be acquired which resulted in minimum energy cost for the rear robot. This information was further transferred to the locomotion controller of the robot to adjust its body posture to save most energy.
The authors studied energy saving in fish school utilizing a bio-inspired robotic fish platform.Specifically, two robotic fish were fixed in the flow tank against the current with constant left-right distance and front-back distance; the body posture of the front robotic fish was controlled to generate dynamic fluid environments, which would affect the rear robotic fish. Based on reinforcement learning, a relationship was established between energy cost and the body posture of the robotic fish. According to this relationship, the body posture could be acquired which resulted in minimum energy cost for the rear robot. This information was further transferred to the locomotion controller of the robot to adjust its body posture to save most energy.
引文
[1]Webb P W.Form and function in fish swimming.Sci Am,1984,251(1):72-82
[2]刘安全,李亮,罗文广,等.一种面向机器鱼的高精度位姿控制算法设计与实现.机器人,2016,38(2):241-247
[3]Liao J C,Beal D N,Lauder G V,et al.The Karman gait:novel body kinematics of rainbow trout swimming in a vortex street.J Exp Biol,2003,206(6):1059-1073
[4]Liao J C.A review of fish swimming mechanics and behaviour in altered flows.Philosophical Transactions of the Royal Society B-Biological Sciences,2007,362:1973-1993
[5]Liao J C,Beal D N,Lauder G V,et al.Fish exploiting vortices decrease muscle activity.Science,2003,302:1566-1569
[6]Mchenry M J,Michel K B,Stewart W,et al.Hydrodynamic sensing does not facilitate active drag reduction in the golden shiner(Notemigonus crysoleucas).J Exp Biol,2010,213(8):1309-1319
[7]Yanase K,Herbert N A,Montgomery J C.Disrupted flow sensing impairs hydrodynamic performance and increases the metabolic cost of swimming in the yellowtail kingfish,Seriola lalandi.J Exp Biol,2012,215(22):3944-3954
[8]Wang C,Xie G,Wang L,et al.CPG-based locomotion control of a robotic fish:using linear oscillators and reducing control parameters via PSO.International Journal of Innovative Computing Information and Control,2011,7(7B):4237-4249
[9]Li L,Wang C,Xie G.A general CPG network and its implementation on the microcontroller.Neurocomputing,2015,167:299-305
[10]周志华.机器学习.北京:清华大学出版社,2016
[11]Kaelbling L P,Littman M L,Moore A W.Reinforcement learning:an introduction.IEEE Transactions on Neural Networks,2005,16(1):285-286
[12]Gosavi A.Reinforcement learning:a tutorial survey and recent advances.Informs Journal on Computing,2009,21(2):178-192
[13]Breder C.On the survival value of fish schools.Zoologica,1967,52(2):25-40
[14]Smits A J,Lim T T.Flow visualization,techniques and examples.London:Imperial College Press,2000
[15]Donley J M,Dickson K A.Swimming kinematics of juvenile kawakawa tuna(Euthynnus affinis)and chub mackerel(Scomber japonicus).Journal of Experimental Biology,2000,203:3103-3106
[16]Huang K Z,Yang H,King I,et al.Machine learning.Advanced Topics in Science&Technology in China,2008,22(3):329-350
[17]Watkins C J C H,Dayan P.Technical note:Q-learning.Machine Learning,1992,8(3/4):279-292
[18]Li L,Jia L,Xie G.Using robotic fish to explore the hydrodynamic mechanism of energy saving in a fish school[EB/OL].(2015-08-13)[2018-05-10].https://arxiv.org/abs/1508.03249
[19]Yu J,Wen L,Ren Z.A survey on fabrication,control,and hydrodynamic function of biomimetic robotic fish.Science China Technological Sciences,2017,60(9):1365-1380
[20]Maertens A P,Triantafyllou M S,Yue D K P.Efficiency of fish propulsion.Bioinspiration Biomimetics,2015,10(4):046013
[21]Proske U,Gandevia S C.The proprioceptive senses:their roles in signaling body shape,body position and movement,and muscle force.Physiol Rev,2012,92(4):1651-1697
[22]Li L,Liu A,Wang W,et al.Bottom-level motion control for robotic fish to swim in groups:modeling and experiments.Bioinspir Biomim,2019,14:046001
[2]刘安全,李亮,罗文广,等.一种面向机器鱼的高精度位姿控制算法设计与实现.机器人,2016,38(2):241-247
[3]Liao J C,Beal D N,Lauder G V,et al.The Karman gait:novel body kinematics of rainbow trout swimming in a vortex street.J Exp Biol,2003,206(6):1059-1073
[4]Liao J C.A review of fish swimming mechanics and behaviour in altered flows.Philosophical Transactions of the Royal Society B-Biological Sciences,2007,362:1973-1993
[5]Liao J C,Beal D N,Lauder G V,et al.Fish exploiting vortices decrease muscle activity.Science,2003,302:1566-1569
[6]Mchenry M J,Michel K B,Stewart W,et al.Hydrodynamic sensing does not facilitate active drag reduction in the golden shiner(Notemigonus crysoleucas).J Exp Biol,2010,213(8):1309-1319
[7]Yanase K,Herbert N A,Montgomery J C.Disrupted flow sensing impairs hydrodynamic performance and increases the metabolic cost of swimming in the yellowtail kingfish,Seriola lalandi.J Exp Biol,2012,215(22):3944-3954
[8]Wang C,Xie G,Wang L,et al.CPG-based locomotion control of a robotic fish:using linear oscillators and reducing control parameters via PSO.International Journal of Innovative Computing Information and Control,2011,7(7B):4237-4249
[9]Li L,Wang C,Xie G.A general CPG network and its implementation on the microcontroller.Neurocomputing,2015,167:299-305
[10]周志华.机器学习.北京:清华大学出版社,2016
[11]Kaelbling L P,Littman M L,Moore A W.Reinforcement learning:an introduction.IEEE Transactions on Neural Networks,2005,16(1):285-286
[12]Gosavi A.Reinforcement learning:a tutorial survey and recent advances.Informs Journal on Computing,2009,21(2):178-192
[13]Breder C.On the survival value of fish schools.Zoologica,1967,52(2):25-40
[14]Smits A J,Lim T T.Flow visualization,techniques and examples.London:Imperial College Press,2000
[15]Donley J M,Dickson K A.Swimming kinematics of juvenile kawakawa tuna(Euthynnus affinis)and chub mackerel(Scomber japonicus).Journal of Experimental Biology,2000,203:3103-3106
[16]Huang K Z,Yang H,King I,et al.Machine learning.Advanced Topics in Science&Technology in China,2008,22(3):329-350
[17]Watkins C J C H,Dayan P.Technical note:Q-learning.Machine Learning,1992,8(3/4):279-292
[18]Li L,Jia L,Xie G.Using robotic fish to explore the hydrodynamic mechanism of energy saving in a fish school[EB/OL].(2015-08-13)[2018-05-10].https://arxiv.org/abs/1508.03249
[19]Yu J,Wen L,Ren Z.A survey on fabrication,control,and hydrodynamic function of biomimetic robotic fish.Science China Technological Sciences,2017,60(9):1365-1380
[20]Maertens A P,Triantafyllou M S,Yue D K P.Efficiency of fish propulsion.Bioinspiration Biomimetics,2015,10(4):046013
[21]Proske U,Gandevia S C.The proprioceptive senses:their roles in signaling body shape,body position and movement,and muscle force.Physiol Rev,2012,92(4):1651-1697
[22]Li L,Liu A,Wang W,et al.Bottom-level motion control for robotic fish to swim in groups:modeling and experiments.Bioinspir Biomim,2019,14:046001