Optimizing Deep Q-Networks with fuzzy inference-based adaptive replay buffer management

Deep reinforcement learning algorithms, such as Deep Q-Networks (DQN), require careful tuning of replay memory parameters. In standard DQN implementations, these parameters remain fixed, which conflicts with the dynamic nature of the learning process where environmental conditions and reward stability continuously change. This mismatch often results in unstable learning or slow convergence. In this paper, we present a fuzzy logic-based system for adaptively adjusting three key replay memory parameters: memory size, the ratio of recent samples, and priority weight. The proposed fuzzy system evaluates the agent’s state by monitoring reward variations and average training errors, and accordingly updates these parameters to maintain optimal values during training. To assess the effectiveness of the proposed approach, we compared it with conventional DQN and PER-DQN methods across three benchmark reinforcement learning environments: CartPole v۱, LunarLander v۲, and Taxi v۳. Experimental and statistical analyses demonstrate that our method improves average rewards, reduces training time, and enhances learning stability.

کلیدواژه ها:

Reinforcement Learning ، deep neural networks ، replay memory ، Fuzzy logic ، adaptive parameter tuning