\documentclass[11pt]{article}
\pdfsuppressptexinfo=-1
\usepackage[margin=1in]{geometry}
\usepackage[T1]{fontenc}
\usepackage{booktabs,array,amsmath,amssymb,hyperref,xcolor,graphicx}
\usepackage[numbers]{natbib}
\hypersetup{pdftitle={Do Self-Improving Coding Agents Beat Textbook Baselines? Evidence from a Darwin G\"odel Machine Port and Two Published Systems},pdfauthor={Tejasvi Singh Tomar}}
\input{numbers.tex}
\input{numbers_v2.tex}
\input{numbers_v3.tex}
\title{Do Self-Improving Coding Agents Beat Textbook Baselines?\\ Evidence from a Darwin G\"odel Machine Port and Two Published Systems}
\author{Tejasvi Singh Tomar\\[2pt]
\normalsize ORCID: \href{https://orcid.org/0000-0003-4668-3639}{https://orcid.org/0000-0003-4668-3639}}
\date{}
\begin{document}
\maketitle

\input{sec_abstract_v2.tex}

\section{Introduction}
A self-improving agent harness lets a coding agent edit the code around a frozen model, such as its prompts, tools and
control flow, and keeps the edits that score better on a benchmark. The Darwin G\"odel Machine (DGM)
\citep{zhang2026dgm} reports SWE-bench rising from 20\% to 50\% this way, and related systems search over agent designs
(ADAS, Automated Design of Agentic Systems \citep{hu2025adas}), guide the search with lineage statistics \citep{wang2026hgm}, or make the improvement procedure
itself editable \citep{zhang2026hyperagents}. Recent audits show that such gains shrink on held-out tasks and against
compute-matched baselines \citep{hu2026winners,wang2026rethinking,ding2026hda,oved2026tokens}, and that a setting
change can match what a search finds \citep{xu2026harnessrepair}. What a self-improvement loop finds that direct
engineering would not find remains an open question.

We address it in two ways. First, in a controlled port of the DGM to hard competitive-programming problems from LiveCodeBench, we compare
the agents that loops select with four baselines: standard engineering changes applied without search (textbook
moves), a single redesign by a language model without evaluation feedback (a one-shot redesign), loops whose
diagnoses, a model's analysis of a failed task, are replaced by requests drawn from a fixed list of generic improvement
requests (random-diagnosis loops), and placebo loops whose children are exact copies of their parents. Second, we rebuild the DGM's own released SWE-bench agents (the initial agent and the reported best)
and run them on SWE-bench Verified tasks the search never saw, next to the initial agent with textbook moves. We
also run the agents released with ADAS under a current model. All three rounds of experiments were pre-registered: a first round on the loops themselves, a second round with the baselines, generality checks and published systems, and a third round that removes confounds the first two left. Changes made after each pre-registration,
among them the choice of the second-round SWE-bench model after pilot runs whose outcomes were known, are summarized in
Appendix~\ref{app:dev}, and the full log is part of the release.

Each baseline removes one ingredient of a loop. Textbook moves are the changes an engineer would make without any
search (ours were written while reading the logs of pilot runs; Appendix~\ref{app:pilot}); if they match a loop's best agent, the search found nothing that an engineer could not match without it. A one-shot
redesign keeps the language model's knowledge and the failure logs but drops the evaluation feedback and the iteration.
Random-diagnosis loops keep the search but replace the diagnosis. Placebo loops keep the selection on a small benchmark
but make no edits, so they show how much of a reported gain selection alone produces.

\paragraph{Findings.} \input{sec_findings_v2.tex}

\input{sec_setup_v2.tex}
\input{sec_round1_v2.tex}
\input{sec_methods_v2.tex}
\input{sec_results_v2.tex}
\input{sec_published_v2.tex}
\input{sec_related_v2.tex}
\input{sec_discussion_v2.tex}

\clearpage
\section*{Code and data availability}
Code, the three pre-registrations, the change log and per-task results are available at
\url{https://doi.org/10.5281/zenodo.23720890}; the same files are on GitHub at
\url{https://github.com/tejasvi/self-improving-agents-textbook-baselines} (tag v4).

\section*{Acknowledgements}
Generative AI tools were used to write code, run analyses and draft text; the author directed the work and takes full responsibility for its content.

\bibliographystyle{plainnat}
\bibliography{refs_v2}

\clearpage
\appendix
\input{sec_appendix_v2.tex}
\end{document}