JAF: Judge Agent Forest

This paper introduces JAF: Judge Agent Forest, a novel framework designed to enhance the self-refinement and evaluation processes of agentic AI systems. Instead of assessing responses in isolation, the judge agent performs joint inference across multiple query-response pairs from a primary agent. This approach allows the judge to become a holistic learner, identifying cross-instance patterns and inconsistencies, and providing aggregate feedback that significantly improves the primary agent's outputs. JAF integrates principles from belief propagation and ensemble learning to create a robust system for context-sensitive judgments, with potential for real-world application in developing more reliable and efficient AI agents.