Learning a model-free robotic continuous state-action task through contractive Q-network

The main purpose of this paper is to make it easier to movetoward Autonomous learning robots using reinforcementlearning (RL). Deterministic policy gradient algorithm ischosen for work in model-free and continuous state-spaceconfiguration. Neural network is chosen as FunctionApproximator (FA) for actor and critic in the algorithm. Anovel method called contractive Q-network is proposed forupdating the critic FA (Q-network). Since this methodrequires fewer samples in learning tasks, it is more efficientto be used in this context. To show the efficiency of thedeveloped method, two illustrative examples are conducted,first in the well-known puddle world and then in PushRecovery (PR) task on a simulated humanoid robot. Therobot learns how to recover from a variation of forcedirections and magnitudes.