Reinforcement learning-based demand-side management by smart charging of electric vehicles

Ozcelik M. B., Kesici M., Aksoy N., Genç V. M. İ.

ELECTRICAL ENGINEERING, cilt.104, sa.6, ss.3933-3942, 2022 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 104 Sayı: 6
  • Basım Tarihi: 2022
  • Doi Numarası: 10.1007/s00202-022-01597-2
  • Dergi Adı: ELECTRICAL ENGINEERING
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Compendex, INSPEC, DIALNET
  • Sayfa Sayıları: ss.3933-3942
  • Anahtar Kelimeler: Electric vehicles, Smart charging, Demand-side management, Markov decision process, Reinforcement learning, Q-learning, Expected SARSA
  • İstanbul Teknik Üniversitesi Adresli: Evet

Özet

In the future, the load demand due to charging of large numbers of electric vehicles (EVs) will be at such a high level that existing networks in some regions may not afford. Therefore, radical changes modernizing the grid will be required to overcome the technical and economic problems besides bureaucratic issues. Amendments to be made in the regulations on electrical energy and new tariff regulations can be considered within this scope. Smart charging of EVs is not often dealt with a solution using reinforcement learning (RL), which is one of the most effective methods for solving such decision-making problems. Most of the studies on this topic endeavor to estimate the state and action spaces and to tune the penalty coefficients within the RL models developed. In this paper, we solve the EV charging problem using expected SARSA with a novel rewarding strategy, as we propose a new approach to determine the state and action spaces. The efficacy of the proposed method is demonstrated on the problem of charging a single EV, as we compare it with a number of alternatives involving Q-Learning and constant charging approaches.