基于Q学习的能量自适应路由算法Routing algorithm of energy adaptive based on Q-learning
黄庆东;张淼;袁润芝;陈晨;
摘要(Abstract)：
为了提高Ad hoc网络的生命周期,提出一种基于Q学习的能量自适应路由算法。交换邻居节点间的信息,将节点剩余能量以及邻居节点中最大剩余能量信息反馈给数据发送的源节点,使源节点进行路由策略的学习,并通过调节学习率的变化平衡网络中路径探索和利用的关系。在数据包进行传输时,将数据包传输给具有最优动作值的下一跳节点,使得节点做出合理的路由决策。仿真结果表明,该算法能够降低网络节点能量损失,提高网络负载均衡,延长网络的生命周期,同时降低路由时延。
关键词(KeyWords)： Q学习;Ad hoc网络;能耗均衡;路由算法;自适应
基金项目(Foundation): 国家科技重大专项项目(2017ZX03001012-005);; 陕西省重点科技创新团队计划项目(2017KCT-30-02);; 陕西省创新人才推进计划-物联网科技创新团队项目(2019TD-028)
作者(Author): 黄庆东;张淼;袁润芝;陈晨;
Email:
DOI: 10.13682/j.issn.2095-6533.2020.04.008
参考文献(References)：
- [1] PANDEY J,HEGDE M.Low-latency and energy-balanced data transmission over cognitive small world WSN[J].IEEE Transactions on Vehicular Technology,2018,67(8):7719-7733.DOI:10.1109/TVT.2018.2839562.
- [2] 梁青,张文飞,上官艺伟,等.无人机自组网中AODV路由协议的改进[J].西安邮电大学学报,2018,23(6):44-49.DOI:10.13682/j.issn.2095-6533.2018.06.009.html.
- [3] SHRUTHI S.Proactive routing protocols for a MANET:A review[C]//Proceedings of the 2017 International Conference on I-SMAC (IoT in Social,Mobile,Analytics and Cloud) (I-SMAC).India Palladam:IEEE,2017:821-827.DOI:10.1109/I-SMAC.2017.8058294.
- [4] 牛玉刚,陈文广.一种基于网格的兼顾拥塞避免与能耗均衡的WSN路由算法[J].控制与决策,2016,31(11):1985-1990.DOI:10.13195/j.kzyjc.2015.1341.html.
- [5] 杜力凯,张灵,陈云华.一种改进的ZigBee路由优化算法[J].计算机科学,2015,42(5):153-156.DOI:10.11896/j.issn.1002-137X.2015.05.030.
- [6] HENDRIKS T,CAMELO M,LATRE S.Q2-routing:A qos-aware Q-routing algorithm for wireless Ad hoc networks[C]//Proceedings of the 2018 14th International Conference on Wireless and Mobile Computing,Networking and Communications (WiMob).Cyprus Limassol:IEEE,2018:108-115.DOI:10.1109/WiMOB.2018.8589161.
- [7] MAMMERI Z.Reinforcement learning based routing in networks:Review and classification of approaches[J].IEEE Access,2019,7:55916-55950.DOI:10.1109/ACCESS.2019.2913776.
- [8] HE X,JIANG H,SONG Y,et al.Routing selection with reinforcement learning for energy harvesting multi-hop CRN[J].IEEE Access,2019,7:54435-54448.DOI:10.1109/ACCESS.2019.2912996.
- [9] KAVALEROV M,LIKHACHEVA Y,SHILOVA Y.A reinforcement learning approach to network routing based on adaptive learning rates and route memory[C]//Proceedings of the IEEE SoutheastCon 2017.Concord:IEEE,2017:1-6.DOI:10.1109/SECON.2017.7925316.
- [10] KAVALEROV M V,SHILOVA Y A,BEZUKLADNIKOV I I.Preventing instability in full echo Q-routing with adaptive learning rates[C]//Proceedings of the IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering.Petersburg:IEEE,2017:155-159.DOI:10.1109/EIConRus.2017.7910517.
- [11] 沙鑫磊,白光伟,张杰,等.双学习率自适应的Q路由算法[J].小型微型计算机系,2019,40(8):1672-1677.DOI:10.3969/j.issn.1000-1220.2019.08.018.
- [12] KAVALEROV M,SHILOVA Y,LIKHACHEVA Y.Adaptive Q-routing with random echo and route memory[C]// Proceedings of the Conference of Open Innovations Association.Petersburg:IEEE,2017:138-145.DOI:EIConRus.2017.7910517.
- [13] LIU Y,LU D,ZHANG G,et al.Q-Learning based content placement method for dynamic cloud content delivery networks[J].Access IEEE,2019,7:66384-66394.DOI:10.1109/ACCESS.2019.2917564.
- [14] MAYADUNNA H,SILVA S L D,WEDAGE I,et al.Improving trusted routing by ldentifying malicious nodes in a MANET using reinforcement learning[C]//Proceedings of the 2017 Seventeenth International Conference on Advances in ICT for Emerging Regions.Sri Lanka Colombo:IEEE,2017:1-8.DOI:10.1109/ICTER.2017.8257821.
- [15] SARKAR S,DATTA R.Mobility-aware route selection technique for mobile Ad hoc networks[J].IET Wireless Sensor Systems,2017,7(3):55-64.DOI:10.1049/iet-wss.2016.0058.