A Small-Scale System for Autoregressive Program Synthesis Enabling Controlled Experimentation

This paper presents Cadmus, a system designed for research on program synthesis with small models, avoiding the complexities and high computational demands of large language models (LLMs). Cadmus includes an integer virtual machine (VM), a dataset of true programs for diverse tasks, and an autoregressive transformer model trainable for under $200. It enables effective and affordable fine-grained control for studying program completion, out-of-distribution representations, inductive reasoning, and instruction following, outperforming GPT-5 with 100% accuracy on a simple arithmetic task while providing transparency into the dataset's relationship to the problem.