Deep Reinforcement Learning for Real-Time Energy Management in Nano-Grids through DQN Approach for Cost and Grid Dependency Reduction

Abstract

Effective control of energy in nano-grids is necessary for achieving energy independence, cutting energy expenses and making use of renewable energy. A new DQN-based reinforcement learning approach is presented in this study for the management of energy in AI-connected smart grids. The state space in the MDP represents the PV output, the load demand, battery SOC and changing electricity rates of the time instant. The possible strategies included in this work are divided into five interpretable groups: charging, discharging, interacting with the grid and scheduling energy output. When reward functions are designed well, they consider the variables of cost, ageing batteries and load satisfaction. The DQN uses experience replay, a target Q-network and exploration with a gradually decreasing epsilon. Experiments were done using real solar power and energy consumption information. Analyzing against nine other models — rule-based logic, MPC, fuzzy logic EMS and hybrid reinforcement learning systems — for eight main metrics, the DQN outperformed all other models. Very importantly, the DQN attained better results, reducing energy cost by 5.13 USD/day, using 94.28% renewables, achieving 91.93% efficiency, LPSP of only 1.32% and reaching Q-value convergence fast in 439 steps. The outcomes reveal that the suggested model can adapt to changing conditions and is suitable for modern smart grid operations.