ScholarWorks@성균관대학교

초록

Self-acquisition of robot skills by reinforcement learning (RL) is a powerful paradigm. However, due to task complexity, RL may take an unacceptable amount of time to converge or even fail to learn. The emerging curriculum and recovery-focused RL can be promising approaches to handle this issue. In this paper, we investigated how integrating curriculum and recovery-focused RL can help obtain a complex robotic skill: the peg-in-hole insertion skill. To this end, we implemented a recovery-focused dynamic curriculum learning within the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework. For recovery, failures are emulated by positioning the peg with substantial lateral and angular misalignments, incurring frequent contact with the environment. For curriculum learning, three task difficulty regimes - Easy, Medium, and Hard - are introduced by progressively increasing offsets in lateral, tilt, and depth deviations. Furthermore, a success-rate-adaptive scheduler dynamically rebalances Easy, Medium, and Hard initial states while preserving a small portion of easier cases to mitigate catastrophic forgetting. Simulation studies indicate that, compared with no-curriculum and static-curriculum baselines, the proposed recovery-focused dynamic curriculum TD3 achieves improved target success rates with fewer environment interactions and higher held-out performance across difficulty levels, demonstrating its effectiveness in robust recovery and efficient policy learning for complex robotic skills.

키워드