\section{Changes after pre-registration}\label{app:dev}
The three pre-registrations and the full log of changes (DEVIATIONS.md) are in the release; this is a summary. The
pre-registrations call the engineered seed the strong or hand-written seed (\texttt{seed\_strong} in the released files).
\begin{enumerate}\setlength{\itemsep}{1pt}\renewcommand{\labelenumi}{(\arabic{enumi})}
\item The first launch was aborted because proxy restarts dropped model calls.
\item The first grader shadowed the built-in \texttt{pow}; the campaign was relaunched with a corrected grader, and five
old-grader loops were kept for the grader analysis.
\item Exploratory comparisons added after earlier results: the loops without hint prompts, the cap-only seed, the
compute-matched resampling baseline and its follow-up for L, the reversal of the output cap in loop 3's agent, the
Sonnet redesigns, the loops
and textbook moves with Claude Haiku~4.5, the textbook moves, one-shot redesigns and resampled seed under gpt-oss-120b,
the first-round transfer runs under Claude Sonnet~5.5, Gemini~2.5 Flash and gpt-oss-120b, the ADAS agents with the best
test score, and the SWE-bench sensitivity analysis. Each was specified before it was computed.
\item The rule that chooses a best candidate on $F_1$ and compares it on $F_2$ (H6, H8) was fixed after the second
pre-registration, while the grid batch was running (about three quarters done) and before any result was summarized
(we had only glanced at the first 100 runs for crashes);
the literal reading is reported as well.
\item H6, H8 and the L comparison pair agents from different batches. The pre-registration's same-batch seeds ``to
control drift'' and the pre-results rule that comparisons use same-batch references point to seed-adjusted contrasts,
while the pre-results rule that names loop 3's agent re-evaluated ``in the same period'' as the comparator fits raw
ones; none of
them says explicitly how such contrasts are formed. The analysis code, written before the comparator's batch finished,
computed raw contrasts, and seed-adjusted contrasts were added after the raw contrasts had been computed. Verdicts are given under both; they differ
only for the literal reading of H6, which fails under the seed-adjusted contrast.
\item H2: the round-2 summary over six loops is given with the registered per-loop task bootstrap and a hierarchical
bootstrap; an earlier analysis used a bootstrap over loop means only.
\item The random-diagnosis list of the second round was written after the first round had identified the empty-turn
failure (\S\ref{sec:round2}); the third round replaced the requests aimed at it (\S\ref{sec:random-clean}).
\item Pilot overlap with the fresh problems: Appendix~\ref{app:pilot}.
\item Proxy spending caps were raised during the runs; evolved agents get one fresh run per problem in the first round;
the improver swap used 12 diagnoses instead of 16.
\item H9 could not be run as registered for loops 2 and 3, because half (loop 2) or all but one (loop 3) of the
leave-one-out variants do not apply. H3 and H5 are computed on $F_1$ instead of $F$, because the single edits and the
transfer runs were evaluated on $F_1$ only; H5 covers loops 1 and 3, because loop 2's agent was not run under the other
models (to limit cost). H4 is reported for every arm started from the engineered seed.
\item In the registered SWE-bench test of the DGM, Claude Haiku~4.5 was replaced by Claude Sonnet~4.5 after a pilot in which most runs of both DGM
agents ended early; the outcomes of a 15-task pilot with Claude Sonnet~4.5 (Appendix~\ref{app:swe}), including resolve
rates, were known when the model and protocol of the full run were fixed; the
registered Claude Haiku~5.5 runs were not made; the third textbook move is a workflow prompt
rather than a loop change; the retry arm costs less than the best agent instead of the same; tasks whose reference
patch fails today were excluded for all agents; hidden-test residue is removed from the agent's container after setup.
\item The registered test of HGM's released agent was not run, because the agent supports only OpenAI's Responses API,
and the registered test of the DGM on its Polyglot benchmark was not run.
\item Part of one exploratory batch and the selection runs of three loops near their end ran during heavy load on the
shared machine (re-grading that batch's saved solutions changed 5 of 1{,}624 outcomes); later batches ran on a separate
machine. During the second-round SWE-bench run the machine's CPU was oversubscribed (per-run mean load up to \vloadMax{} on 64
cores). Load was sampled every minute over the whole run in \vloadFullPct\% of runs and over part of it in
\vloadPartPct\%, and interpolated from checks every 30 minutes for the rest, including \vloadMovesApprox{} of the
textbook agent's 293 runs. By load tertile (low, middle, high), the
early-ending rates were: initial agent \vearlyInitLo, \vearlyInitMid{} and \vearlyInitHi\%; best agent \vearlyBestLo,
\vearlyBestMid{} and \vearlyBestHi\%; textbook agent \vearlyMovesLo, \vearlyMovesMid{} and \vearlyMovesHi\%; retry
variant \vearlyRetryLo, \vearlyRetryMid{} and \vearlyRetryHi\%. The resolve rate of the textbook agent fell from
\vresMovesLo\% to \vresMovesHi\%, which works against it. The tertiles are formed per
agent and are confounded with run order, so these figures do not rule out an effect of load. In the third round the
one-minute load average stayed below \wLoadMax{} (in the second round it reached \wvLoadMax), and the best agent still
ended \wBestEarly{} of \wN{} runs early, compared with \wvBestEarly{} on the same tasks in the second round (\S\ref{sec:swe-equal}).
\item Third round: the loops ran with eight task workers each instead of twelve (parallelism only) on a spot machine
that was preempted three times during the runs, twice during the loops and once during the third-round batch. Each time the
jobs resumed from their saved state, and an unfinished child or run was redone. The spending cap of that machine was
raised by \$40 before the third-round batch, and its third-round spend was \$331.66, against the \$330 registered. The
SWE-bench machine reached the internet only through address translation, and the requirement files that the swebench
code reads from GitHub were served from a local copy of the same files. The third-round SWE-bench run used eight task containers per agent,
half the earlier parallelism. Comparisons with the second round's SWE-bench runs on the same tasks are exploratory, because they cross
batches. In the third-round SWE-bench run, the early endings of the best agent were \wBestEarlySig{} on a signal and \wBestEarlyOne{} with exit code 1,
all those of the other two agents were on a signal, and \wBestEarlyBoth{} of the best agent's tasks end early in both rounds.
The kernel logged one out-of-memory kill; none of the \wOomInProg{} runs then in progress ended early within ten minutes of
it, and \wOomEarlyLater{} of them ended early at least \wOomGapMin{} minutes later. With the patches of early-ended runs
kept, the three agents resolve \wMovesKept\%, \wInitKept\% and \wBestKept\% (textbook, initial, best), and the initial
agent is \wsMovesInitAbs{} points below the textbook agent ($p=\wsMovesInitP$). In an exploratory comparison of the same
tasks between the second-round batch and this one, the rate of the best agent changes by $\wxBest$ points [$\wxBestlo$,
$\wxBesthi$] and that of the initial agent by $\wxInit$ [$\wxInitlo$, $\wxInithi$], while the textbook agent reproduces
its earlier rate within $\wxMovesAbs$ points; the two batches differ in the output limit, the parallelism and the machine
load.
\item Third round: exploratory analyses first computed during internal review and then added: the second-round
random-diagnosis loops against the DGM loops in the third-round batch, loop 3's agent against L in that batch, the redesigns
that raise the cap with and without logs, the runs in progress at the out-of-memory kill, and the second-round early
endings on the same 65 tasks.
\end{enumerate}

\section{Contrasts across batches}\label{app:batches}
H6, H8 and the comparison of L with one retry pair agents from different batches. Loop 3's agent was re-evaluated in the
ablation batch (\vbtAbl{} UTC), which started together with the grid batch of the 16 combinations (\vbtGrid{} UTC) and
was about a quarter done when the grid ended; the redesign batch ran within the ablation batch's hours (\vbtRed{} UTC).
Relative to the seed in the grid batch, the seed in the ablation batch scores $\vsvAblGridF$ points [$\vsvAblGridFlo$,
$\vsvAblGridFhi$] on all of $F$. Relative to the seed in the redesign batch, it scores $\vsvAblRedFtwo$
[$\vsvAblRedFtwolo$, $\vsvAblRedFtwohi$] on $F_2$ and $\vsvAblRedF$ [$\vsvAblRedFlo$, $\vsvAblRedFhi$] on all of $F$, so
raw contrasts are less favourable to the redesigns than seed-adjusted ones by these amounts. A seed-adjusted contrast is
(candidate minus its batch's seed) minus (comparator minus its batch's seed), paired by problem; item 5 of
Appendix~\ref{app:dev} explains why both are given. Table~\ref{tab:batches} lists both. The verdicts agree except for the
literal reading of H6, which passes on raw scores and fails after seed adjustment.
\begin{table}[h]\centering\small
\caption{Contrasts across batches, raw and seed-adjusted (candidate minus comparator, in points, with 95\%
intervals).}\label{tab:batches}
\begin{tabular}{p{0.46\linewidth}lll}\toprule
contrast & problems & raw & seed-adjusted \\ \midrule
H6: combination chosen on $F_1$ (\vHsixWinner) vs loop 3's agent & $F_2$ & $\vHsixFtwo$ [$\vHsixFtwolo$, $\vHsixFtwohi$] & $\vbaHsix$ [$\vbaHsixlo$, $\vbaHsixhi$] \\
H6, literal: best combination on $F$ (\vhsixLitBest) vs loop 3's agent & $F$ & $\vhsixLit$ [$\vhsixLitlo$, $\vhsixLithi$] & $\vhsixLitAdj$ [$\vhsixLitAdjlo$, $\vhsixLitAdjhi$] \\
all four moves vs loop 3's agent & $F$ & $\vpsEBLPRaw$ [$\vpsEBLPRawlo$, $\vpsEBLPRawhi$] & $\vpsEBLPAdj$ [$\vpsEBLPAdjlo$, $\vpsEBLPAdjhi$] \\
L vs loop 3's agent & $F$ & $\vpsLRaw$ [$\vpsLRawlo$, $\vpsLRawhi$] & $\vpsLAdj$ [$\vpsLAdjlo$, $\vpsLAdjhi$] \\
H8: average Opus redesign vs average loop best & $F$ & $\vHeightOpusmean$ [$\vHeightOpusmeanlo$, $\vHeightOpusmeanhi$] & $\vbaOpusMean$ [$\vbaOpusMeanlo$, $\vbaOpusMeanhi$] \\
H8: best of five Opus redesigns (chosen on $F_1$) vs loop 3's agent & $F_2$ & $\vHeightOpusbest$ [$\vHeightOpusbestlo$, $\vHeightOpusbesthi$] & $\vbaOpusBest$ [$\vbaOpusBestlo$, $\vbaOpusBesthi$] \\
H8, literal: best Opus redesign on $F$ vs loop 3's agent & $F$ & $\vheightLitOpusRaw$ [$\vheightLitOpusRawlo$, $\vheightLitOpusRawhi$] & $\vheightLitOpusAdj$ [$\vheightLitOpusAdjlo$, $\vheightLitOpusAdjhi$] \\
L with one retry vs loop 3's agent & $F$ & $\vltwoRaw$ [$\vltwoRawlo$, $\vltwoRawhi$] & $\vltwoAdj$ [$\vltwoAdjlo$, $\vltwoAdjhi$] \\
\bottomrule\end{tabular}\end{table}

\section{Pilot overlap}\label{app:pilot}
The engineered seed, and with it the textbook moves E, B, L and P, was written while two short pilot runs of the DGM
seed took place, and their logs were read as it was written. The second pilot covered 80 problems, \vnpN{} of which
are in $F$ (and 7 in the selection set); the first used a subset of them. Of the second pilot's \vpilotRuns{} runs,
\vpilotNoSol{} ended without a solution, \vpilotEmptyResp{} of them on an empty response, the failure that L repairs.
L is therefore a repair informed by failure logs, like a one-shot redesign, rather than a move chosen without them. Without those problems, L gains $\vnpLgain$ points [$\vnpLgainlo$,
$\vnpLgainhi$] in the final batch, no loop agent is significantly better than L and $\vnpWorse$ are significantly worse
(best: $\vnpLoopThreeVsL$ [$\vnpLoopThreeVsLlo$, $\vnpLoopThreeVsLhi$]), and the H6 contrast on $F_2$ is $\vnpHsix$
[$\vnpHsixlo$, $\vnpHsixhi$].

\input{sec_othermodels_v3.tex}

\section{Time and budget limits}\label{app:cens}
This appendix gives the shares of runs that end on a signal, reach the wall-clock limit, or spend at least 99\% of the
per-run budget, for the batches in which any of these is common. In the final batch under Claude Haiku~4.5 (\$1
budget), the DGM seed ends on a signal in \vsigKSeed\% of runs (\vsigKSeedTr\% in the transfer runs) and the
engineered seed in \vsigKStrong\%; the engineered seed reaches the budget in \vbudKStrong\%. Loop 1's agent ends on a
signal in \vsigKOne\% of runs and reaches the time limit in \vtoKOne\%. Loop 2's agent ends on a signal in
\vsigKTwo\%, reaches the time limit in \vtoKTwo\% and the budget in \vbudKTwo\%. In the first-round transfer runs,
loop 3's agent reaches the \$1 budget in \vbudTrKThreeN{} of 116 runs under Claude Haiku~4.5 and in \vbudTrSThreeN{}
under Claude Sonnet~4.6, and the DGM seed ends on a signal in \vsigSSeed\% of its Sonnet~4.6 runs. Under gpt-oss-120b
no run ends on a signal, and the share of runs that reach the 600\,s limit ranges from 47 to 100\% across agents in the
final batch, from 14 to 60\% in the textbook-move batch and from 28 to 94\% in the redesign batch. In the Haiku~5.5
final batch at most 1\% of runs end on a signal; time-outs reach 22\% for one loop agent and are under 10\% for every
other agent. Pass rates under these limits are censored from above.

\section{First-round details}\label{app:round1}
\begin{table}[h]\centering\small
\caption{Single edits of the DGM seed and log-free chains (the diagnosis sees only the agent's code, no task log). Fresh gain on $F_1$ averaged over evaluated members, the same
average counting children that failed to compile as unchanged, and the member with the best selection score.}\label{tab:ladder}
\footnotesize\setlength{\tabcolsep}{3pt}
\begin{tabular}{lrrrrrr}\toprule
diagnosis given to the editing step & tried & failed & evaluated & mean gain [95\% CI] & with failures & best-by-$S$ \\ \midrule
agent code only (no task data) & \nladConeatt & \nladConefail & \nladConen & $\nladCone$ [$\nladConelo$, $\nladConehi$] & $\nladConeitt$ & $\nladConebest$ \\
task log, no hint prompts & \nladCthreeatt & \nladCthreefail & \nladCthreen & $\nladCthree$ [$\nladCthreelo$, $\nladCthreehi$] & $\nladCthreeitt$ & $\nladCthreebest$ \\
DGM diagnosis (task log and hint prompts) & \nladCtwoatt & \nladCtwofail & \nladCtwon & $\nladCtwo$ [$\nladCtwolo$, $\nladCtwohi$] & $\nladCtwoitt$ & $\nladCtwobest$ \\
log-free chain 1 (steps) & \nladDoneatt & \nladDonefail & \nladDonen & $\nladDone$ [$\nladDonelo$, $\nladDonehi$] & $\nladDoneitt$ & $\nladDonebest$ \\
log-free chain 2 (steps) & \nladDtwoatt & \nladDtwofail & \nladDtwon & $\nladDtwo$ [$\nladDtwolo$, $\nladDtwohi$] & $\nladDtwoitt$ & $\nladDtwobest$ \\ \bottomrule
\end{tabular}\end{table}
\paragraph{Single edits.} A diagnosis model that sees only the agent's code proposes plausible generic changes; none
of its \nladConeatt{} diagnoses mentions the output cap, stop reasons or empty responses. Implemented by the seed, the
changes gain about two points on average (Table~\ref{tab:ladder}), about the same as edits from a
task log without the DGM's hint prompts. Single DGM steps gain two to three times as much (counting children that
failed to compile as unchanged, or not); the children that came from the empty-patch prompt carry that gain, and all children from the stochasticity prompt failed to produce a working
edit. Log-free chains mostly broke the agent. These single-edit gains, and the two-point gains of loops started from the
engineered seed, are measured against a pooled seed reference whose morning runs scored about 2.5 points higher than
its evening runs, so they depend on the reference to within about that much.

\paragraph{Grader quirk.} Our first grader prepended \texttt{from math import *} to every solution, which replaces the
built-in \texttt{pow} and breaks \texttt{pow(a, b, m)}. Loops from the engineered seed proposed tools that run a
solution under several import preludes. Re-scoring the saved archives with the corrected grader shows that most of
their measured gain (\nquirkStrongMeasText{} on the selection set) was recovery from the bug (\nquirkStrongCorrText{}
corrected); the two graders rank the archives with Spearman $\rho=\nQArho$ and $\nQBrho$.

\paragraph{Improver swap.} The same twelve DGM diagnoses of the seed were implemented on a fresh copy of the seed by
the seed agent and by loop 3's best agent. \nswapText{} Among the evolved agent's finished edits, \nswapLinesText{}. A
better coding agent does not yield more working agents from the same instructions.

\begin{table}[h]\centering\small
\caption{First-round agents under other task models ($F_1$, one run per problem, \$1 budget, \$0.25 for gpt-oss-120b).
Gains are paired against the DGM seed under the same model; budget and time limits censor some agents
(Appendix~\ref{app:cens}).}\label{tab:transfer}
\begin{tabular}{lrrrr}\toprule
task model & DGM seed & engineered seed & loop-3 best: gain & loop-1 best: gain \\ \midrule
Claude Haiku~4.5 & \ntrHfourseed & \ntrHfourstrong & $\ntrHfourthree$ [$\ntrHfourthreelo$, $\ntrHfourthreehi$] & $\ntrHfourone$ [$\ntrHfouronelo$, $\ntrHfouronehi$] \\
Claude Sonnet~4.6 & \ntrSfourseed & \ntrSfourstrong & $\ntrSfourthree$ [$\ntrSfourthreelo$, $\ntrSfourthreehi$] & $\ntrSfourone$ [$\ntrSfouronelo$, $\ntrSfouronehi$] \\
Claude Sonnet~5.5 & \ntrSfiveseed & \ntrSfivestrong & $\ntrSfivethree$ [$\ntrSfivethreelo$, $\ntrSfivethreehi$] & $\ntrSfiveone$ [$\ntrSfiveonelo$, $\ntrSfiveonehi$] \\
Gemini~2.5 Flash & \ntrGemseed & \ntrGemstrong & $\ntrGemthree$ [$\ntrGemthreelo$, $\ntrGemthreehi$] & $\ntrGemone$ [$\ntrGemonelo$, $\ntrGemonehi$] \\
gpt-oss-120b & \ntrOssseed & \ntrOssstrong & $\ntrOssthree$ [$\ntrOssthreelo$, $\ntrOssthreehi$] & $\ntrOssone$ [$\ntrOssonelo$, $\ntrOssonehi$] \\ \bottomrule
\end{tabular}\end{table}

\section{SWE-bench details}\label{app:swe}
\paragraph{Rebuilding the agents.} The released DGM repository starts after the SWE-bench run. Its initial agent
differs from the run-time agent in two places that the archive's patch contexts reveal: a later retry around the model
call and an example line in the editor. With both restored, the six patches of the reported best agent's lineage
apply in order without a rejected hunk. The textbook agents start from the restored initial agent; the retry variant
uses helper functions the initial agent already contains (regression-test discovery, test runs and a report score).

\paragraph{Protocol.} All four agents run through the DGM's harness with one change (each task gets its own proxy token
and the proxy address), on 246 fresh tasks (SWE-bench Verified minus the DGM's 200, minus tasks whose environment does
not build or whose reference patch fails today) and 47 of the DGM's 60 selection tasks, one run per task, \$10 per run,
16 tasks in parallel per agent, all four agents at once in the same task order. A pilot with Claude Haiku~4.5 was
stopped because most runs of both DGM agents ended early; Claude Sonnet~4.5 was used instead. A 15-task pilot with
it showed 7 of the best agent's 14 runs and 3 of the initial agent's 15 runs ending early; the full run kept the same
protocol. Grading follows the DGM's
own report script; evaluations that timed out were re-graded alone. One task (scikit-learn-14710) timed out in
evaluation for both textbook agents even when re-graded alone and counts as unresolved; in the sensitivity analysis one
patch of the best agent (psf\_\_requests-1142) got no report and counts as unresolved.

\paragraph{Early endings.} Of the best agent's \vsweBestEarly{} early endings in the second-round run, \vsweBestEarlySig{} came on
a signal (exit codes 137 and 143) and \vsweBestEarlyOne{} with exit code 1. Of the saved logs of these runs, \vsweKillEmpty{} are
empty; \vsweKillLogs{} of the other \vsweKillNonEmpty{} contain a command that ends processes (\texttt{pkill}, \texttt{killall}
or \texttt{kill -9}), and \vsweKillNone{} contain no such command. In the pilot with Claude Haiku~4.5 we traced such endings to
cleanup commands issued through the agent's own shell. Median run times were \vsweInitMedMin{} minutes for the initial
agent and \vsweRetryMedMin{} for the retry variant; among the runs that finished, the retry variant resolves
$\vsweRetryFinFresh\%$ and the initial agent $\vsweInitFinFresh\%$ of the fresh tasks.

\section{Costs}\label{app:cost}
Model spend was about \$4{,}600 for the competitive-programming study (all three rounds) and the ADAS test together,
and about \$3{,}700 for the SWE-bench tests; machine time was small by comparison. These totals cover the model calls of the
experiments only.