OpenClaw-RL: Train Any Agent Simply by Talking

This framework converts real-time "next-state signals" from AI agent interactions into continuous, online learning sources. It recovers both implicit evaluative signals and explicit directive signals, enabling agents to achieve rapid personalization in conversational settings and improve performance across diverse general agent tasks like terminal, GUI, SWE, and tool-calling environments. This allows agents to improve simply by being used, adapting to user re-queries, corrections, and explicit feedback.

Designing reward functions and configuring RL environments typically requires deep domain expertise and tedious trial-and-error engineering, severely limiting the accessibility, speed, and scalability of RL applications for non-experts.

The framework employs a prompt-to-reward translation pipeline using state-of-the-art LLMs. The user engages in a multi-turn dialogue to specify desired behaviors. The LLM generates Python-based reward functions and environment wrappers, which are compiled and used to train a standard RL algorithm (like PPO) in an iterative feedback loop where behavior adjustments are made conversationally.

Experiments across standard continuous control and discrete environments show that agents trained via conversational rewards achieve parity with manually engineered rewards in 85% of evaluated tasks, simultaneously reducing human setup time by approximately 70%.

While the conversational interface massively lowers the barrier to entry, the system heavily relies on the LLM's coding capabilities. It may fail silently if the generated reward function has subtle logical errors leading to reward hacking. Debugging these generated functions still inevitably requires some degree of expert knowledge.