Article Sidebar

Download
PDF
Statistic
Read Counter : 3019 Download : 170

Main Article Content

Abstract

The demand for better energy technologies has sparked research and development of electric and hybrid vehicles. Due to their clean, sustainable, and high energy density, fuel cell vehicles have begun to stand out above the rest. Therefore, fuel cell hybrid vehicles can compete with internal combustion engine-powered vehicles in the future. However, fuel cells face obstacles, including slow dynamics that necessitate managing their operation together favorably. To reduce an HEV's operational costs, this study analyzes the HEV energy management issue utilizing machine learning approaches, particularly reinforcement learning. This paper aims to comprehensively review the existing work on a couple of machine-learning-based energy management systems for an electric vehicle run by hydrogen fuel cell, it can be concluded that progress was evident when the Q-learning-based algorithm was utilized towards lowering the SOC battery variation of about 0.7 per unit which was the primary task, while a Deep Deterministic Policy Gradient (DDPG) based energy management system (EMS) starts operating the fuel cell at a higher efficiency rate comparatively while using the battery.

Keywords

Fuel cell Battery Renewable energy sources Energy management Machine learning Reinforcement learning Q leaning

Article Details

How to Cite
Habib, A. K. M. R. R. (2023). A comparative study of the machine learning-based energy management system for hydrogen fuel cell electric vehicles. Future Technology, 3(1), 13–24. Retrieved from https://fupubco.com/futech/article/view/103
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
Bookmark and Share

References

  1. T. Smith, C. Raucci, S. H. Hosseinloo, I. Rojon, J. Calleya, S. S.De La Fuente, P. Wu, and K. Palmer, “CO2 emissions from internationalshipping. Possible reduction targets and their associated pathways,”UMAS: London, UK, Tech. Rep., 2016.
  2. Wu, P. (2020). Decarbonising coastal shipping using fuel cells and batteries (Doctoral dissertation, UCL (University College London)), uri: https://discovery.ucl.ac.uk/id/eprint/10101024.
  3. N. Gray, S. McDonagh, R. O’Shea, B. Smyth, and J. D. Murphy,“Decarbonising ships, planes and trucks: An analysis of suitable low-carbon fuels for the maritime, aviation and haulage sectors,”Advancesin Applied Energy, p. 100008, 2021.
  4. N. Sulaiman, M. Hannan, A. Mohamed, P. J. Ker, E. Majlan, and W. W.Daud, “Optimization of energy management system for fuel-cell hybridelectric vehicles: Issues and recommendations,”Applied energy, vol.228, pp. 2061–2079, 2018.
  5. H. Peng, J. Li, L. L ̈owenstein, and K. Hameyer, “A scalable, causal,adaptive en ergy management strategy based on optimal control theoryfor a fuel cell hybrid railw ay vehicle,”Applied Energy, vol. 267, p.114987, 2020.
  6. P. Wu, J. Partridge, and R. Bucknall, “Costeffective reinforcementlearning energy management for plugin hybrid fuel cell and batteryships,”Applied Energy, vol. 275, p. 115258, 2020.
  7. W. Wu, J. Partridge, and R. Bucknall, “Stabilised control strategy forpem fuel cell and supercapacitor propulsion system for a city bus,”International Journal of Hydrogen Energy, vol. 43, no. 27, pp. 12 302–12 313, 2018.
  8. S. Eriksen, M. L ̈utzen, J. B. Jensen, and J. C. Sørensen, “Improving theenergy efficiency of ferries by optimizing the operational practices,” inProceedings of the Full Scale Ship Performance Conference 2018: TheRoyal Institution of Naval Architects.The Royal Institution of NavalArchitects, 2018, pp. 101–111.
  9. S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approxi-mation error in actor-critic methods,”arXiv preprint arXiv:1802.09477,2018.
  10. W. He, L. Tao, L. Han, Y. Sun, P. E. Campana, and J. Yan, “Optimal analysis of a hybrid renewable power system for a remote island,” Renewable Energy, vol. 179, pp. 96–104, 2021.
  11. A. Boretti and S. Castelletto, “Cost and performance of CSP and PV plants of capacity above 100 MW operating in the United States of America,” Renewable Energy Focus, vol. 39, pp. 90–98, 2021.
  12. O. Alves, L. Calado, R. M. Panizio, M. Gonçalves, E. Monteiro, and P. Brito, “Techno-economic study for a gasification plant processing residues of sewage sludge and solid recovered fuels,” Waste Management, vol. 131, pp. 148–162, 2021.
  13. A. Alsharif, C. W. Tan, R. Ayop, A. Dobi, and K. Y. Lau, “A comprehensive review of energy management strategy in vehicle-to-grid technology integrated with renewable energy sources,” Sustainable Energy Technologies and Assessments, vol. 47, article 101439, 2021.
  14. D.-D. Tran, M. Vafaeipour, M. El Baghdadi, R. Barrero, J. Van Mierlo,and O. Hegazy, “Thorough state-of-the-art analysis of electric and hybridvehicle powertrains: Topologies and integrated energy managementstrategies,”R enewable and Sustainable Energy Reviews, vol. 119, p.109596,
  15. J. Wu, H. He, J. Peng, Y. Li, and Z. Li, “Continuous reinforcementlearning of energy management with deep q network for a power splithybrid electric bus,”Applied energy, vol. 222, pp. 799–811, 2018.
  16. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G.Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovskiet al., “Humanlevel control through deep reinforcement learning,”Nature, vol. 518, no. 7540, p. 529, 2015.
  17. P. Wu and R. Bucknall, “Hybrid fuel cell and battery propulsionsystem modelling and multi-objective optimisation for a coastal ferry,”International Journal of Hydrogen Energy, vol. 45, no. 4, pp. 3193–3208, 2020.
  18. “On the design of plugin hybrid fuel cell and lithium batterypropulsion systems for coastal ships,” in13th Int ernational MarineDesign Conference (IMDC 2018), P. Kujala and L. Lu, Eds., vol. 2. London: CRC Press/Balkema, 2018, Conference Proceedings, pp. 941–951.
  19. D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller,“Deterministic policy gradient algorithms,” inProceedings of the 31stInternati onal Conference on Machine Learning, 2014, pp. 387–395.
  20. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa,D. Silver, and D. Wierstra, “Continuous control with deep reinforcementlearning,”arXiv preprint, 2015.
  21. H. van Hasselt, “Double q-learning,” inAdvances in Neural InformationProcessing Systems 23, J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor,R. S. Zemel, and A. Culotta, Eds.Curran Associates, Inc., 2010, pp.2613–2621.
  22. Henderson, Peter & Islam, Riashat & Bachman, Philip & Pineau, Joelle & Precup, Doina & Meger, David. (2017). Deep Reinforcement Learning That Matters. Proceedings of the AAAI Conference on Artificial Intelligence. 32. 10.1609/aaai.v32i1.11694.
  23. N. Sulaiman, M. A. Hannan, A. Mohamed, P. J. Ker, E. H. Majlan, and W. R. Wan Daud, “Optimization of energy management system for fuel-cell hybrid electric vehicles: Issues and recommendations,” Appl. Energy, vol. 228, pp. 2061–2079, Oct. 2018.
  24. H. Li, A. Ravey, A. N’Diaye, and A. Djerdir, “A Review of Energy Management Strategy for Fuel Cell Hybrid Electric Vehicle,” in 2017 IEEE Vehicle Power and Propulsion Conference (VPPC), 2017, pp. 1–6.
  25. J. Han, J.-F. Charpentier, and T. Tang, “An Energy Management System of a Fuel Cell/Battery Hybrid Boat,” Energies, vol. 7, no. 5, pp. 2799–2820, May 2014.
  26. M. K. Zadeh, L. Saublet, R. Gavagsaz-Ghoachani, B. Nahid Mobarakeh, S. Pierfederici, and M. Molinas, “Energy management and stabilization of a hybrid DC microgrid for transportation applications,” in 2016 IEEE Applied Power Electronics Conference and Exposition (APEC), 2016, pp. 3397–3402.
  27. M. K. Zadeh, Stability Analysis Methods and Tools for Power Electronics- Based DC Distribution Systems, Applicable to On-Board Electric Power Systems and Smart Microgrids. NTNU, 2016.
  28. S. N. Motapon, L. Dessaint, and K. Al-Haddad, “A Comparative Study of Energy Management Scemes for a Fuel-Cell Hybrid Emergency Power System of More-Electric Aircraft,” IEEE Trans. Ind. Electron., vol. 61, no. 3, pp. 1320–1334, Mar. 2014.
  29. S. J. Moura, D. S. Callaway, H. K. Fathy, and J. L. Stein, “Tradeoffs between battery energy capacity and stochastic optimal power management in plug-in hybrid electric vehicles,” J. Power Sources, vol. 195, no. 9, pp. 2979–2988, May 2010.
  30. X. Zhang, C. C. Mi, A. Masrur, and D. Daniszewski, “Wavelet transform-based power management of hybrid vehicles with multiple on-board energy sources including fuel cell, battery and ultracapacitor,” J. Power Sources, vol. 185, no. 2, pp. 1533–1543, Dec. 2008.
  31. Chen, Tzu-Chia & Al-Azzawi, Fouad & Grimaldo, John & Chetthamrongchai, Paitoon & Dorofeev, Aleksei & masood, aras & Ahmed, Alim Al Ayub & Akhmadeev, Ravil & Latipah, Asslia & Abu Al-Rejal, Hussein. (2022). Development of Machine Learning Methods in Hybrid Energy Storage Systems in Electric Vehicles. Mathematical Problems in Engineering. 2022. 1-8. 10.1155/2022/3693263.
  32. F. Segura and J. M. Andújar, “Power management based on sliding control applied to fuel cell systems: A further step towards the hybrid control concept,” Appl. Energy, vol. 99, pp. 213–225, Nov. 2012.
  33. G. Paganelli, S. Delprat, T. M. Guerra, J. Rimaux, and J. J. Santin, “Equivalent consumption minimization strategy for parallel hybrid powertrains,” in Vehicular Technology Conference. IEEE 55th Vehicular Technology Conference. VTC Spring 2002 (Cat. No.02CH37367), 2002, vol. 4, pp. 2076–2081 vol.4.
  34. J. Han, Y. Park, and Y. Park, “A novel updating method of equivalent factor in ECMS for prolonging the lifetime of battery in fuel cell hybrid electric vehicle,” IFAC Proc. Vol., vol. 45, no. 30, pp. 227–232, Jan. 2012.
  35. A. Chatzivasileiadi, E. Ampatzi, and I. Knight, “Characteristics of electrical energy storage technologies and their applications in buildings,” Renewable and Sustainable Energy Reviews, vol. 25, pp. 814–830, 2013.
  36. O. Z. Sharaf and M. F. Orhan, “An overview of fuel cell technology: fundamentals and applications,” Renewable and Sustainable Energy Reviews, vol. 32, pp. 810–853, 2014.
  37. J. Han, Y. Park, and D. Kum, “Optimal adaptation of equivalent factor of equivalent consumption minimization strategy for fuel cell hybrid electric vehicles under active state inequality constraints,” J. Power Sources, vol. 267, pp. 491–502, Dec. 2014.
  38. P. García, J. P. Torreglosa, L. M. Fernández, and F. Jurado, “Viability study of a FCbattery-SC tramway controlled by equivalent consumption minimization strategy,” Int. J. Hydrog. Energy, vol. 37, no. 11, pp. 9368–9382, Jun. 2012.
  39. C. Zheng, S. W. Cha, Y. Park, W. S. Lim, and G. Xu, “PMP-based power management strategy of fuel cell hybrid vehicles considering multi-objective optimization,” Int. J. Precis. Eng. Manuf., vol. 14, no. 5, pp. 845–853, May 2013.
  40. C. H. Zheng, G. Q. Xu, Y. I. Park, W. S. Lim, and S. W. Cha, “Prolonging fuel cell stack lifetime based on Pontryagin’s Minimum Principle in fuel cell hybrid vehicles and its economic influence evaluation,” J. Power Sources, vol. 248, pp. 533–544, Feb. 2014.
  41. T. Li, H. Liu, and D. Ding, “Predictive energy management of fuel cell supercapacitor hybrid construction equipment,” Energy, vol. 149, pp. 718–729, Apr. 2018.
  42. J. P. Torreglosa, P. García, L. M. Fernández, and F. Jurado, “Predictive Control for the Energy Management of a Fuel-Cell–Battery–Supercapacitor Tramway,” IEEE Trans. Ind. Inform., vol. 10, no. 1, pp. 276–285, Feb. 2014.
  43. M.-J. Kim and H. Peng, “Power management and design optimization of fuel cell/battery hybrid vehicles,” J. Power Sources, vol. 165, no. 2, pp. 819–832, Mar. 2007.
  44. W.-S. Lin and C.-H. Zheng, “Energy management of a fuel cell/ultracapacitor hybrid power system using an adaptive optimal control method,” J. Power Sources, vol. 196, no. 6, pp. 3280– 3289, Mar. 2011.
  45. S. Mohan, Y. Kim, and A. G. Stefanopoulou, “Estimating the Power Capability of Li-ion Batteries Using Informationally Partitioned Estimators,” IEEE Trans. Control Syst. Technol., vol. 24, no. 5, pp. 1643–1654, Sep. 2016.
  46. R. C. Hsu, S. Chen, W. Chen, and C. Liu, “A Reinforcement Learning Based Dynamic Power Management for Fuel Cell Hybrid Electric Vehicle,” in 2016 Joint 8th International Conference on Soft Computing and Intelligent Systems (SCIS) and 17th International Symposium on Advanced Intelligent Systems (ISIS), 2016, pp. 460–464.
  47. F.-D. Li, M. Wu, Y. He, and X. Chen, “Optimal control in microgrid using multi-agent reinforcement learning,” ISA Trans., vol. 51, no. 6, pp. 743–751, Nov. 2012.
  48. P. Kofinas, S. Doltsinis, A. I. Dounis, and G. A. Vouros, “A reinforcement learning approach for MPPT control method of photovoltaic sources,” Renew. Energy, vol. 108, pp. 461– 473, Aug.2017.
  49. R. Liessner, C. Schroer, A. Dietermann, and B. Bäker, “Deep Reinforcement Learning for Advanced Energy Management of Hybrid Electric Vehicles,” presented at the International Conference on Agents and Artificial Intelligence, 2018, vol. 2, pp. 61–72.
  50. M. Prauzek, N. R. A. Mourcet, J. Hlavica, and P. Musilek, “Q Learning Algorithm for Energy Management in Solar Powered Embedded Monitoring Systems,” in 2018 IEEE Congress on Evolutionary Computation (CEC), 2018, pp. 1–7.
  51. S. Kim and H. Lim, “Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings,” Energies, vol. 11, no. 8, p. 2010, Aug. 2018.
  52. S. Etemad and N. Mozayani, “Using reinforcement learning to make smart energy storage sources in microgrid,” in 2015 30th International Power System Conference (PSC), 2015, pp. 345–350.
  53. C. Liu, “Optimal Power Management Based on Q-Learning and Neuro-Dynamic Programming for Plug-in Hybrid Electric Vehicles,” PhD Dissertation, University of Michigan, Dearborn, Michigan, USA, 2018.
  54. R. C. Hsu, C. T. Liu, and D. Y. Chan, “A reinforcementlearning-based assisted power management with QoR provisioning for human–electric hybrid bicycle,” IEEE Transactions on Industrial Electronics, vol. 59, no. 8, pp. 3350–3359, 2011.
  55. X. Qi, G. Wu, K. Boriboonsomsin, M. J. Barth, and J. Gonder “Data-driven reinforcement learning-based real-time energy management system for plug-in hybrid electric vehicles,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2572, no. 1, pp. 1–8, 2016.
  56. C. Liu and Y. L. Murphey, “Power management for plug-in hybrid electric vehicles using reinforcement learning with trip information,” in Proceedings of the 2014 IEEE Transportation Electrification Conference and Expo (ITEC), pp. 1–6, IEEE, Dearborn, MI, USA, June 2014.
  57. T. Liu, Y. Zou, D. Liu, and F. Sun, “Reinforcement learning of adaptive energy management with transition probability for a hybrid electric tracked vehicle,” IEEE Transactions on Industrial Electronics, vol. 62, no. 12, pp. 7837–7846, 2015.
  58. J. Hu, D. Liu, C. Du, F. Yan, and C. Lv, “Intelligent energy management strategy of hybrid energy storage system for electric vehicle based on driving pattern recognition,” Energy, vol. 198, p. 117298, 2020.
  59. A. A. Kamel, H. Rezk, and M. A. Abdelkareem, “Enhancing the operation of fuel cell-photovoltaic-battery-supercapacitor renewable system through a hybrid energy management strategy,” International Journal of Hydrogen Energy, vol. 46, no. 8, pp. 6061–6075, 2021.
  60. P. Zhao, Y. Wang, N. Chang, Q. Zhu, and X. Lin, “A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles,” in Proceedings of the 2018 23rd Asia and South Pacific design automation conference (ASP-DAC), pp. 196–202, IEEE, Jeju, South Korea, January 2018.
  61. R. Xiong, J. Cao, and Q. Yu, “Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle,” Applied Energy, vol. 211, pp. 538–548, 2018.
  62. H. Yu, M. Kuang, and R. McGee, “Trip-oriented energy management control strategy for plug-in hybrid electric vehicles,” in Proceedings of the 2011 50th IEEE Conference on Decision and Control and European Control Conference, pp. 1323–1336, Orlando, FL, USA, January 2013.
  63. X. Lin, Z. Wang, and J. Wu, “Energy management strategy based on velocity prediction using back propagation neural network for a plug-in fuel cell electric vehicle,” International Journal of Energy Research, vol. 45, no. 2, pp. 2629–2643, 2021.
  64. N. P. Reddy, D. Pasdeloup, M. K. Zadeh and R. Skjetne, "An Intelligent Power and Energy Management System for Fuel Cell/Battery Hybrid Electric Vehicle Using Reinforcement Learning," 2019 IEEE Transportation Electrification Conference and Expo (ITEC), 2019, pp. 1-6, doi: 10.1109/ITEC.2019.8790451.
  65. S. Kim and H. Lim, “Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings,” Energies, vol. 11, no. 8, p. 2010, Aug. 2018.
  66. Y. Wu, H. Tan, J. Peng, H. Zhang, and H. He, “Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus,” Applied energy, vol. 247, pp. 454–466, 2019.
  67. Eğer, Z. E. (2022). Reinforcement learning based energy management strategy for fuel cell hybrid vehicles (Doctoral dissertation), uri: https://research.sabanciuniv.edu/id/eprint/42949.