Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Tree

Optimizing scientific computing algorithms for modern GPUs is a labor-intensive and iterative process involving repeated code modification, benchmarking, and tuning across complex hardware and software stacks. This paper introduces a novel framework that leverages large language models (LLMs) to automate and enhance this optimization process. We propose PhyloEvolve, an LLM-agent system that reframes GPU-oriented algorithm optimization as an In-Context Reinforcement Learning (ICRL) problem, enabling trajectory-conditioned reuse of optimization experience without model retraining. The system uses a phylogenetic tree representation to organize optimization history and demonstrates consistent improvements in runtime, memory efficiency, and correctness on scientific computing workloads.