Digital-Twin-Assisted Safe Reinforcement Learning for Resilient Active Distribution Network Operation Under Renewable Generation and Electric Vehicle Uncertainty

The increasing integration of photovoltaic generation, battery energy storage systems, and electric vehicles is transforming conventional distribution networks into active and highly dynamic energy systems. Although these technologies provide substantial flexibility, their stochastic behavior introduces significant challenges for real-time network operation. Renewable generation varies with weather conditions, electricity demand changes over time, and electric-vehicle charging behavior depends on uncertain arrival times, departure times, and state-of-charge requirements. In addition, distribution networks are increasingly exposed to disruptive events such as equipment failures and extreme weather conditions. This paper proposes a digital-twin-assisted safe reinforcement learning framework for resilient operation of active distribution networks under renewable-generation and electric-vehicle uncertainty. The proposed framework integrates a physics-based digital twin, uncertainty-aware scenario generation, continuous-control reinforcement learning, and an action-level safety mechanism. The digital twin provides a virtual representation of the distribution network and enables large numbers of operating and contingency scenarios to be generated without requiring proprietary real-world measurements. A Soft Actor-Critic agent is employed to coordinate battery energy storage and electric-vehicle charging decisions. A safety layer evaluates the actions generated by the RL agent and prevents potentially infeasible decisions from being directly applied to the network. The training environment incorporates stochastic variations in load demand, photovoltaic generation, electric-vehicle availability, battery state of charge, and network contingencies. The resulting controller is designed to operate under both normal and disturbed conditions. Its performance is evaluated using operational cost, voltage security, branch loading, peak demand, electric-vehicle charging satisfaction, energy-not-served, recovery time, and robustness under distribution shifts. The proposed methodology is designed around an IEEE ۳۳-bus active distribution network containing photovoltaic generation, battery energy storage, and electric-vehicle charging facilities. Four control configurations are considered to isolate the effects of conventional reinforcement learning, safety enforcement, and uncertainty-aware training. Particular attention is given to scenarios that are not observed during training, allowing the generalization capability of the proposed controller to be investigated. The main contribution of this work is not the introduction of another standalone reinforcement-learning algorithm, but the development of an integrated and reproducible framework combining digital twins, safe reinforcement learning, uncertainty-aware training, and resilience-oriented evaluation. The proposed framework can be implemented entirely through simulation and therefore provides a practical methodology for studying intelligent control of future active distribution networks without requiring access to proprietary field data.

کلیدواژه ها:

Active distribution network ، digital twin ، safe reinforcement learning ، Soft Actor-Critic ، resilience ، electric vehicles ، battery energy storage ، photovoltaic generation ، uncertainty ، smart grid