基于多智能体深度强化学习的船舶协同避碰策略.pdfVIP

基于多智能体深度强化学习的船舶协同避碰策略

摘要：船舶避碰是智能航行中首要解决的问题，多船会遇局面下，只有相互协作,共同规划避碰策略，才能有效降

低碰撞风险.为使船舶智能避碰策略具有协同性、安全性和实用性,提出一种基于多智能体深度强化学习的船舶

协同避碰决策方法.首先，研究船舶会遇局面辨识方法，设计满足《国际海上避碰规则》的多船避碰策略.其次，研

究多船舶智能体合作方式，构建多船舶智能体协同避碰决策模型：利用注意力推理方法提取有助于避碰决策的关

键数据；设计记忆驱动的经验学习方法，有效积累交互经验；引入噪音网络和多头注意力机制，增强船舶智能体决

策探索能力.最后，分别在实验地图与真实海图上，对多船会遇场景进行仿真实验.结果表明，在协同性和安全性

方面,相较于多个对比方法,所提出的避碰策略均能获得具有竞争力的结果,且满足实用性要求，从而为提高船舶

智能航行水平和保障航行安全提供一种新的解决方案.

关键词：多智能体深度强化学习；多智能体通信模型；多智能体合作；协同决策；船舶避碰；协同避碰策略

Shipcooperativecollisionavoidancestrategybasedonmulti-agentdeep

reinforcementlearning

Abstract:Shipcollisionavoidanceistheprimaryissueinintelligentnavigation.Inmulti-shipencounters,onlyby

collaboratingandjointlyplanningcollisionavoidancestrategies,thecollisionriskcanbeeffectivelyreduced.Inorder

tomaketheshipintelligentcollisionavoidancestrategycollaborative,safeandpractical,ashipcollaborativecollision

avoidancedecisionmethodbasedonmulti-agentdeepreinforcementlearningisproposed.Firstly,themethodof

identifyingshipencountersituationsisstudiedandamulti-shipcollisionavoidancestrategythatsatisfiestheInternational

regulationsforpreventingcollisionsatseaisdesigned.Secondly,byanalysingthecooperationmodeofmulti-shipagents,

amulti-shipagentcooperativecollisionavoidancedecision-makingmodelisconstructed.Themodelusestheattention

inferencemethodtoextractthekeydatathatishelpfulforcollisionavoidancedecisions.Andamemorydrivenexperience

learningmethodisdesignedtoeffectivelyaccumulateinteractiveexperience.Inaddition,thenoisenetworkandmulti-

headattentionmechanismareintroducedintothemodeltoenhancedecision-makingandexplorationcapabilitiesof

shipagents.Finally,ontheexperimentalmapandtherealnauticalchart,simulationexperimentsarecarriedouton

themulti-shipencounterscenarios.Theresultsshowthatintermsofcollaboration