AI-Powered Annotation Pipelines for Stabilizing Large Language Models: A Human-AI Synergy Approach

This paper introduces an AI-based annotation pipeline designed to systematically identify, label, and fix instability patterns in Large Language Model (LLM) output. This human-AI synergy method combines automated weak supervision and confidence-based annotation with targeted human validation to improve the reliability and ethical soundness of feedback information, addressing LLM instability issues in highly regulated industries.