Abstract

Abstract—This paper presents a reinforcement learning (RL) framework for controlling a planar bipedal robot with nine degrees of freedom (DOF), incorporating prismatic joints in both the shank and thigh segments. The inclusion of prismatic knee and thigh joints allows the robot to dynamically adjust its leg length during locomotion, significantly enhancing adaptability to uneven terrain and improving walking stability. The control architecture is implemented within a custom Gymnasium environment, leveraging the MuJoCo physics engine for high-fidelity simulation of the robot’s dynamics, contact interactions, and frictional effects. The RL policy is trained using state-of-the-art algorithms, with observation inputs comprising joint positions and velocities, ground reaction forces, and friction estimates. Simulation results demonstrate that the bipedal robot can successfully navigate uneven terrain without falling throughout the evaluated episode length, while ensuring all joint torque commands remain within the specified actuator limits.