Recursive self-improvement

The idea is a loop: a system changes its own code or design, and if the change makes it a more capable designer, its next change can go further. Eliezer Yudkowsky described it in 2000: “Smarter and smarter AIs become more and more capable of rewriting their own code and making themselves even smarter.” He also noted that the loop could stall if a change made the system faster without making it smarter. Running an optimizing compiler over its own source code may produce a faster compiler, but repeating the procedure “accomplishes nothing”.

Today’s practice of models helping to train or evaluate models is narrower. In Constitutional AI, a model critiques and revises its own answers against a written list of principles; a model then compares pairs of answers, and a preference model trained on those comparisons supplies the reward signal for further training (Bai et al., 2022). Using a strong model to grade other models’ answers is called LLM-as-a-judge (Zheng et al., 2023). In Self-Rewarding Language Models, a model grades its own answers to supply its own rewards during training (Yuan et al., 2024). In these methods researchers design the training process, and the model’s critiques, comparisons or grades are inputs to it; Yuan et al. ran three rounds of their method.

I. J. Good described the idea in Speculations Concerning the First Ultraintelligent Machine (Advances in Computers, volume 6). He defined an ultraintelligent machine as one that can far surpass all the intellectual activities of any person. Since designing machines is one of those activities, such a machine could design still more capable machines, and “there would then unquestionably be an ‘intelligence explosion’”. He also wrote that a machine “could gradually improve itself out of all recognition”. The volume carries a 1965 copyright (Virginia Tech record), while the publisher’s metadata dates it 1966 (Crossref record). An early use of the exact phrase recursive self-improvement is in Yudkowsky’s Coding a Transhuman AI 2.0a (copyright 2000, Internet Archive capture of 17 August 2000), which describes a “seed AI” built to improve itself.