Uncertainty-Aware Proximal Policy Optimization for Dynamic Spatio-Temporal Graphs

Dynamic network optimization in spatio-temporal random graphs presents significant challenges due to inherent stochasticity and temporal dependencies. In this study, we introduce an uncertainty-aware proximal policy optimization (UA-PPO) framework that explicitly incorporates probabilistic modeling of uncertainty into reinforcement learning for dynamic graph optimization. By integrating temporal graph structures with stochastic node and edge features, the proposed method leverages policy gradients regularized by uncertainty estimates, allowing for robust decision-making under dynamic and partially observed network conditions. We evaluate the framework across benchmark datasets with variable node connectivity and temporal evolution patterns, demonstrating improved convergence, stability, and expected cumulative reward compared to standard PPO and deep Q-learning baselines. Additionally, we provide quantitative analyses using performance tables, ablation studies, and probabilistic confidence intervals to validate the robustness of our approach. Our findings suggest that explicitly accounting for uncertainty in spatio-temporal random graphs significantly enhances the adaptability and reliability of reinforcement learning policies for real-world dynamic network problems.

کلیدواژه ها:

Dynamic network optimization ، Spatio-temporal random graphs ، Proximal policy optimization (PPO) ، Probabilistic reinforcement learning ، Uncertainty modeling ، Temporal graph neural networks ، Stochastic decision-making