SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

Large Language Model (LLM) agents struggle to learn from past experiences, with existing memory methods often storing redundant trajectories and failing to extract high-level patterns. SkillRL addresses this by bridging raw experience and efficient policy improvement through automatic skill discovery and recursive skill evolution. It builds a hierarchical SkillBank, uses an adaptive retrieval strategy, and incorporates a recursive evolution mechanism, significantly reducing token footprint and enhancing reasoning utility. Experimental results on ALFWorld, WebShop, and search-augmented tasks demonstrate state-of-the-art performance, outperforming strong baselines by over 15.3% and maintaining robustness as task complexity increases.