\begin{partabstract} We study the spherical perceptron with Gaussian patterns at negative margins, a mean-field model of jamming in which the set of solutions is not convex. We prove the Parisi formula for its free energy for bounded-above $C^2$ activations with bounded second derivative and either bounded first derivative or concavity, without any monotonicity assumption on the activation. For every constraint density $\alpha>2$, we show that the normalized logarithm of the volume of solutions converges in probability to the Parisi value at every margin below a critical margin $\kappa_{\mathrm c}(\alpha)<0$, that this value tends to $-\infty$ continuously as the margin increases to $\kappa_{\mathrm c}(\alpha)$, and that the largest achievable margin converges in probability to $\kappa_{\mathrm c}(\alpha)$. At sufficiently negative margins, Gardner's replica-symmetric formula for the free energy is exact. For $\alpha$ slightly above two, the limit $\kappa_{\mathrm c}(\alpha)$ lies below Gardner's prediction by at least a constant times $(\alpha-2)^3$; with Part~II, this order is exact. At the optimal margin, with probability exponentially close to one, every optimal configuration has exactly $N$ contacts: the jamming point is isostatic. The empirical gap measures of optimal configurations, normalized by $N$, converge weakly in probability to a deterministic measure with an atom of mass exactly one at zero. At a fixed margin below $\kappa_{\mathrm c}(\alpha)$, the gap statistics of all but an exponentially small fraction of the feasible set are given by the Parisi diffusion. \end{partabstract} \part{Parisi formula, capacity, and isostaticity for the negative spherical perceptron} \chapter{Introduction}\label{I:sec:intro} The perceptron is the simplest model of a neural network and a basic example of a random constraint satisfaction problem. Given $M$ random patterns $g_1,\dots,g_M\in\R^N$ and a margin $\kappa\in\R$, the spherical perceptron asks for a vector $\sigma$ on the sphere of radius $\sqrt N$ such that \[ \frac{g_a\cdot\sigma}{\sqrt N}\ge\kappa\qquad\text{for every }a\le M . \] Two questions are classical: how large the set of such vectors is, and what the largest achievable margin is when $M/N\to\alpha$. Gardner and Derrida computed the answers with the replica method, under the assumption of replica symmetry \cite{Gardner1988,GardnerDerrida1988}. In particular, Gardner's formula predicts that the largest achievable margin converges to the solution $\kRS(\alpha)$ of \[ \alpha\,\E(\kRS-G)_+^2=1, \] where $G$ is a standard Gaussian variable. For $\kappa\ge0$ each constraint cuts out a geodesically convex spherical cap, and replica symmetry is expected to hold throughout the satisfiable phase. In this regime Gardner's formula for the free energy was proved by Shcherbina and Tirozzi \cite{ShcherbinaTirozzi2003} and by Talagrand \cite[Chapter~8]{TalagrandVolII}, and Stojnic gave another proof of the formula for the capacity \cite{Stojnic2013Gardner}. For $\kappa<0$, which is the relevant regime when $\alpha>2$, each constraint removes a cap smaller than a hemisphere, and the feasible set is a nonconvex intersection of the complements of such caps. Franz and Parisi proposed this \emph{negative spherical perceptron} as the simplest model of jamming \cite{FranzParisi2016}. Franz, Parisi, Sevelev, Urbani, and Zamponi argued that its satisfiability threshold belongs to the same universality class as the jamming transition of hard spheres in infinite dimension \cite{FPSUZ2017}. The same model describes linear classification with a negative margin, a basic example of an overparametrized learning problem \cite{FranzHwangUrbani2019,MontanariZhongZhou2024}. In the physical picture, replica symmetry is lost at negative margins before the satisfiability threshold is reached. The stability of the replica-symmetric solution of the perceptron was studied in \cite{GardnerDerrida1988,Bouten1994}. Close to $\kappa=0$ replica symmetry is lost at a de Almeida--Thouless line \cite{AlmeidaThouless1978}, through a continuous transition to full replica symmetry breaking in the sense of Parisi \cite{Parisi1979,Parisi1980Sequence}, and the whole jamming line lies in the broken phase \cite{FranzParisi2016,FPSUZ2017}. The physics literature therefore predicts that Gardner's formula for the critical margin is exact only for $\kappa\ge0$. At the critical margin, jammed configurations are predicted to be \emph{isostatic}: they are in contact with as many constraints as there are degrees of freedom. Franz and Parisi predicted that jamming is hypostatic in the convex regime $\kappa>0$ and isostatic in the nonconvex regime $\kappa<0$ \cite{FranzParisi2016}; for the isostatic counting in jammed packings see \cite{Alexander1998,Moukarzel1998}. The gaps and contact forces of jammed configurations are predicted to have power-law distributions with universal exponents \cite{CKPUZ2014,CKPUZ2014JSM,FranzParisi2016,FPSUZ2017,ParisiUrbaniZamponi2020}, related to marginal stability \cite{Wyart2012,ParisiZamponi2026} and to the jamming of soft spheres \cite{OHern2003}. Annesi, Malatesta, and Zamponi computed the satisfiability threshold numerically from the equations of full replica symmetry breaking, and located the region of the phase diagram in which the overlap distribution of typical solutions has connected support \cite{AnnesiMalatestaZamponi2025}. Rigorous results for negative margins have so far been partial. In \cite{Stojnic2013Gardner}, Stojnic used Gordon's comparison inequality \cite{GordonY1985} to show that Gardner's formula gives an upper bound on the capacity at every margin. He also proved a sharper bound by applying Gordon's inequality to an exponential of the cost; its improvement over the replica-symmetric bound was shown numerically \cite{Stojnic2013}. El Alaoui and Sellke conjectured that the Parisi variational problem for the hard-margin perceptron, in the form used below, gives the free energy and the capacity \cite[Conjecture~1]{ElAlaouiSellke2022}; they trace this conjecture to \cite{FPSUZ2017}. Assuming that some Parisi minimizer $\gamma$ has no overlap gap, and given oracle access to it, their algorithm constructs vectors of norm close to $\sqrt{Q_\gamma N}$ with a small vector of constraint violations, where $Q_\gamma<1$ is the top of the support of $\gamma$ \cite[Theorem~1]{ElAlaouiSellke2022}; see also \cite{AlaouiMontanariSellke2021}. Montanari, Zhong, and Zhou proved upper and lower bounds on the capacity that agree to leading order as $\kappa\to-\infty$, and studied its algorithmic tractability \cite{MontanariZhongZhou2024}. Their upper bound \cite[Theorem~4.2]{MontanariZhongZhou2024} shows that Gardner's prediction overestimates the capacity when $\alpha$ is large. Near $\alpha=2$, where $\kRS(\alpha)$ is close to zero, the cited bounds do not quantify this separation. Stojnic later proposed a characterization of the capacity through fully lifted random duality theory, and computed the resulting values numerically at finite levels of a stationarized lifting \cite{Stojnic2023}. Huang, Sellke, and Sun characterized the empirical distributions of the margins that algorithms with dimension-free Lipschitz dependence on the Gaussian data can achieve \cite{HuangSellkeSun2026}. For the Ising perceptron, where $\sigma\in\{-1,1\}^N$, the capacity predicted by Krauth and M\'ezard \cite{KrauthMezard1989} was studied by Ding and Sun and by Huang, who proved lower and upper bounds matching the prediction, subject in part to numerical conditions \cite{DingSun2025,Huang2024}. Xu proved a sharp threshold for the Ising perceptron with Bernoulli disorder \cite{Xu2021}. At small constraint densities, Talagrand proved that the free energy of the Ising perceptron is given by the replica-symmetric formula, for a class of activations that includes the one-sided threshold \cite{Talagrand2000HalfSpaces}, \cite[Chapter~2]{TalagrandVolI}, \cite[Chapter~9]{TalagrandVolII}; his treatment of the Shcherbina--Tirozzi model, whose spins are continuous, is in \cite[Chapter~3]{TalagrandVolI}. Bolthausen, Nakajima, Sun, and Xu gave another proof of this replica-symmetric formula, for a larger class of activations \cite{BolthausenNakajimaSunXu2022}. In the Bayes-optimal (teacher--student) setting, replica-symmetric formulas for perceptron-type generalized linear models were established by Barbier, Krzakala, Macris, Miolane, and Zdeborov\'a \cite{BarbierEtAl2019}. The Parisi formula for mean-field spin glasses was predicted in \cite{Parisi1979,Parisi1980Sequence} (see also \cite{mezard1987spin,charbonneau2023spin}) and proved by Guerra and Talagrand for the Sherrington--Kirkpatrick model \cite{Guerra2003,Talagrand2006}. Talagrand and W.-K.\ Chen proved the Crisanti--Sommers formula \cite{CrisantiSommers1992} for spherical models, the latter through the Aizenman--Sims--Starr scheme \cite{Talagrand2006Spherical,Chen2013,AizenmanSimsStarr2003}, and Panchenko proved the formula for general mixed $p$-spin models, building on the ultrametricity theorem \cite{Panchenko2013Ultrametricity,Panchenko2014Parisi,PanchenkoBook}. The perceptron differs from these models in a basic way. Its Hamiltonian $\sum_aU(g_a\cdot\sigma/\sqrt N)$ is a nonlinear function of Gaussian fields, and it is not itself a Gaussian process in $\sigma$. Structurally the model is closer to multi-species or bipartite spin glasses, with one species formed by the coordinates and the other by the constraints. For such models Guerra's interpolation gives an upper bound when the interaction is convex \cite{BarraContucciMingioneTantari2015,Panchenko2015,BatesSohn2022EJP}. The matching lower bound by the Aizenman--Sims--Starr scheme requires synchronization of the overlaps of the species \cite{Panchenko2015,Panchenko2018Vector}, and it needs no convexity, for Ising spins \cite{Panchenko2015} and for spherical spins \cite{BatesSohn2022EJP}; see also \cite{BatesSohn2022CMP,BatesSohn2025}. The interaction of the bipartite model, which the perceptron resembles, is not convex. For spherical bipartite models see \cite{AuffingerChen2014Bipartite,Subag2023Bipartite,Kivimae2023}. Mourrat proved that the solution of an infinite-dimensional Hamilton--Jacobi equation bounds the free energy of bipartite models from above, and conjectured that it is the limit \cite{Mourrat2021}; see also \cite{DominguezMourrat2024}. Our upper bound follows the proof of the upper bound in \cite{Mourrat2021}. That work also contains a quantitative synchronization estimate for pairs of overlap arrays, which is an input to both bounds below. Mourrat later proved an upper bound of the same kind for vector spin glasses whose energy is a Gaussian field \cite{Mourrat2023Vector}; the energy $\sum_aU(g_a\cdot\sigma/\sqrt N)$ is not of this form. For nonconvex vector models, H.-B.\ Chen and Mourrat showed that the limit of the free energy, if it exists, is a critical value of a Parisi-type functional \cite{ChenMourrat2025}, and H.-B.\ Chen proved the same for nonconvex multi-species models \cite{HBChen2026}. H.-B.\ Chen, Issa, and Mourrat identified the free energy of nonconvex multi-species models with centered Ising spins \cite{ChenIssaMourrat2026}. H.-B.\ Chen and Mourrat then identified the limit of the free energy of all multi-species spherical spin glasses, with no convexity assumption \cite{ChenMourrat2026Spherical}. Ho proved that the enriched free energy of vector spin glasses converges without convexity, so the critical-point representation of H.-B.\ Chen and Mourrat holds without assuming convergence \cite{Ho2026}. Our contribution is a Parisi formula for the free energy, an identification of the limit of the largest achievable margin, and a description of the gap statistics and contacts of optimal configurations for the negative spherical perceptron. We prove the Parisi formula for smooth activations, including activations that are neither monotone nor concave (\Cref{I:thm:parisi}). For $\alpha>2$ we identify the exponential volume of the feasible set at every margin below the critical margin $\kc(\alpha)$ of the Parisi functional, and we prove that the largest achievable margin converges to $\kc(\alpha)$ (\Cref{I:thm:capacity}). For $\alpha$ slightly above two, we show that $\kc(\alpha)$ lies below Gardner's prediction by at least a constant times $(\alpha-2)^3$ (\Cref{I:thm:gardner}). Combined with the matching upper bound on $\kRS-\kc$ (Part~II, \Cref{II:thm:thresholds}), this order is exact (\Cref{I:cor:thresholds}). Our main result on the geometry of solutions concerns the configurations of maximal margin. With probability exponentially close to one, every optimal configuration has exactly $N$ contacts. All configurations whose margin is close to optimal share a deterministic limiting distribution of gaps, and this distribution has an atom of mass exactly one at zero (\Cref{I:thm:gaps}). Thus, in the limit, no macroscopic set of constraints beyond the $N$ contacts accumulates near contact. This is a rigorous form of the predicted isostaticity of the jamming point recalled above. The link between the random model and the limiting gap law is a collapse of Parisi minimizers. As the margin increases to $\kc(\alpha)$, the minimal value of the Parisi functional tends to $-\infty$ continuously, and the mass of every minimizer (defined in \Cref{I:sec:model}) tends to zero at a power rate (\Cref{I:thm:jamming}). This is a rigorous counterpart of the physical picture of jamming, in which the overlap of solutions tends to one and the weight of the overlaps below any fixed level tends to zero \cite{FranzParisi2016}. \section{Model and the Parisi functional}\label{I:sec:model} Fix $\alpha>0$ and integers $M=M_N$ with $M_N/N\to\alpha$. Let $\bG=(g_{ai})_{a\le M,\,i\le N}$ have independent standard Gaussian entries, write $g_a$ for its $a$-th row, and put $b_a=g_a/\sqrt N$. Let $\mu_N$ be the uniform probability measure on \[ S_N=\{\sigma\in\R^N:\norm\sigma^2=N\}. \] For $\sigma\in S_N$, the number $b_a\cdot\sigma$ is the \emph{margin} of the constraint $a$ at $\sigma$; for each fixed $\sigma\in S_N$ it is a standard Gaussian variable. For an activation $U:\R\to[-\infty,\infty)$ that is bounded above, define \begin{equation}\label{I:eq-model} \begin{split} Z_N(U)&=\int_{S_N}\exp\Big\{\sum_{a=1}^MU(b_a\cdot\sigma)\Big\}\,\mu_N(\dd\sigma),\\ f_N(U)&=\frac1N\log Z_N(U),\qquad F_N(U)=\E f_N(U). \end{split} \end{equation} For a margin $\kappa\in\R$, the feasible set, its volume, and the largest achievable margin are \begin{equation}\label{I:eq-hard-model} \begin{split} S_N(\kappa)&=\{\sigma\in S_N:b_a\cdot\sigma\ge\kappa\text{ for all }a\le M\},\\ V_N(\kappa)&=\mu_N(S_N(\kappa)), \qquad \kappa_N=\max_{\sigma\in S_N}\min_{a\le M}b_a\cdot\sigma . \end{split} \end{equation} We use the convention $\log0=-\infty$. Thus $V_N(\kappa)=Z_N(H_\kappa)$ for the hard constraint $H_\kappa(x)=\log\1_{\{x\ge\kappa\}}$. For $\sigma\in S_N$ let $m_N(\sigma)=\min_{a\le M}b_a\cdot\sigma$, so that $\kappa_N=\max_{S_N}m_N$. The \emph{gaps} of $\sigma$ are the numbers $b_a\cdot\sigma-m_N(\sigma)\ge0$, and the \emph{empirical gap measure} of $\sigma$ is \[ \nu_{N,\sigma}=\frac1N\sum_{a=1}^M\delta_{\,b_a\cdot\sigma-m_N(\sigma)}, \] a measure on $[0,\infty)$ of total mass $M/N$. A constraint with gap zero is called a \emph{contact} of $\sigma$. An \emph{order parameter} is a nondecreasing, right-continuous function $\gamma:[0,1]\to[0,1]$ that is identically one on $[\bar q,1]$ for some $\bar q<1$. We write $\cU$ for the set of order parameters and \[ Q_\gamma=\inf\{q\in[0,1]:\gamma(q)=1\}<1, \qquad \lambda_\gamma(q)=\int_q^1\gamma(t)\dd t . \] Thus $\gamma$ is the distribution function of a probability measure $\mu_\gamma$ on $[0,Q_\gamma]$, and $Q_\gamma$ is the top of its support. We call $\lambda_\gamma(0)$ the \emph{mass} of $\gamma$. Given $\gamma\in\cU$ and an activation $U$, let $u_\gamma=u_\gamma(q,x;U)$ solve the \emph{Parisi equation} \begin{equation}\label{I:eq-pde} \partial_qu_\gamma+\frac12\partial_{xx}u_\gamma +\frac{\gamma(q)}2(\partial_xu_\gamma)^2=0, \qquad u_\gamma(1,x;U)=U(x). \end{equation} When $\gamma$ is a step function, \eqref{I:eq-pde} is solved by an explicit backward recursion of Gaussian integrals, the Cole--Hopf transformation \cite{Hopf1950,Cole1951} on each step; for general $\gamma$, and for the terminal data used in this part, the solution is defined by approximation. Both are recalled in \Cref{I:sec:functional}. For the Sherrington--Kirkpatrick model these constructions are in \cite{Parisi1980Sequence,Duplantier1981,Guerra2001,Guerra2003} and \cite{JagannathTobasco2016}. The \emph{Crisanti--Sommers entropy} \cite{CrisantiSommers1992,Talagrand2006Spherical} and the \emph{Parisi functional} are \begin{equation}\label{I:eq-parisi} \begin{split} \CS(\gamma)&=\frac12\int_0^{\bar q}\frac{\dd q}{\lambda_\gamma(q)} +\frac12\log(1-\bar q),\\ \cP_U(\gamma)&=\alpha u_\gamma(0,0;U)+\CS(\gamma), \qquad \cP_*(U)=\inf_{\gamma\in\cU}\cP_U(\gamma), \end{split} \end{equation} where $\bar q$ is any number in $[Q_\gamma,1)$. The value of $\CS(\gamma)$ does not depend on this choice, since $\lambda_\gamma(q)=1-q$ on $[Q_\gamma,1]$. The first term in $\cP_U(\gamma)$ accounts for the constraints and the second for the spherical entropy. For the hard constraint, let $\phi$ and $\Phi$ denote the standard Gaussian density and distribution function. For $\gamma\in\cU$, $\bar q\in[Q_\gamma,1)$ and $\kappa\in\R$, let $u_\gamma(q,x;\kappa)$ solve \eqref{I:eq-pde} on $[0,\bar q]\times\R$ with the terminal datum \begin{equation}\label{I:eq-hard-terminal} u_\gamma(\bar q,x;\kappa)=\log\Prob\bigl\{x+\sqrt{1-\bar q}\,G\ge\kappa\bigr\} =\log\Phi\Big(\frac{x-\kappa}{\sqrt{1-\bar q}}\Big), \end{equation} where $G$ is a standard Gaussian variable, and set \begin{equation}\label{I:eq-kappac} \begin{split} \cP_\kappa(\gamma)&=\alpha u_\gamma(0,0;\kappa)+\CS(\gamma),\qquad \cP_*(\kappa)=\inf_{\gamma\in\cU}\cP_\kappa(\gamma),\\ \kc(\alpha)&=\sup\{\kappa\in\R:\cP_*(\kappa)>-\infty\}. \end{split} \end{equation} Again the value does not depend on $\bar q\in[Q_\gamma,1)$. The terminal datum \eqref{I:eq-hard-terminal} is what one obtains by running the equation with $U=H_\kappa$ from time $1$ to time $\bar q$ with $\gamma=1$, which avoids a singular terminal problem. This variational problem appears in the physics literature \cite{FranzParisi2016,FPSUZ2017}, and it was stated in the form \eqref{I:eq-hard-terminal}--\eqref{I:eq-kappac} by El Alaoui and Sellke \cite{ElAlaouiSellke2022}. For the terminal datum \eqref{I:eq-hard-terminal}, they constructed a classical solution of \eqref{I:eq-pde}, with bounds on its derivatives, and proved that its derivatives depend continuously on $\gamma$ \cite[Section~4]{ElAlaouiSellke2022}. We call $\cP_*(\kappa)$ the \emph{Parisi value} and $\kc(\alpha)$ the \emph{critical margin}. The replica-symmetric order parameters $\1_{[q,1]}$, $q\in[0,1)$, give Gardner's replica-symmetric functional: \begin{equation}\label{I:eq-RS} \PRS_\kappa(q)=\cP_\kappa(\1_{[q,1]}) =\alpha\E\log\Phi\Big(\frac{\sqrt q\,G-\kappa}{\sqrt{1-q}}\Big) +\frac q{2(1-q)}+\frac12\log(1-q). \end{equation} Recall that $\kRS(\alpha)$ is the unique solution of $\alpha\E(\kRS-G)_+^2=1$. It is Gardner's prediction for the limit of $\kappa_N$ \cite{Gardner1988}, and $\kRS(\alpha)<0$ if and only if $\alpha>2$, since $\E G_-^2=\frac12$. The value $\alpha=2$ at $\kappa=0$ is Cover's capacity of the perceptron \cite{Cover1965}. Finally, for $\gamma\in\cU$ and $\kappa\in\R$ we define the \emph{Parisi diffusion} $X=X^{\gamma,\kappa}$ as follows. On $[0,Q_\gamma]$ it is the unique strong solution of \begin{equation}\label{I:eq-diffusion} \dd X_q=\gamma(q)\,\partial_xu_\gamma(q,X_q;\kappa)\dd q+\dd B_q, \qquad X_0=0, \end{equation} for a standard Brownian motion $B$. Conditionally on $(X_q)_{q\le Q_\gamma}$, the endpoint $X_1$ has the law of $X_{Q_\gamma}+\sqrt{1-Q_\gamma}\,G$ conditioned on the event $\{X_{Q_\gamma}+\sqrt{1-Q_\gamma}\,G\ge\kappa\}$, where $G$ is a standard Gaussian variable independent of $B$. The equation \eqref{I:eq-diffusion} is well posed by \Cref{I:pf:apriori,I:pf:coef-cont}. For mixed $p$-spin models this diffusion appears in the work of Auffinger and W.-K.\ Chen \cite[Theorem~3 and~(17)]{AuffingerChen2015Uniqueness}, and for the terminal datum \eqref{I:eq-hard-terminal} it appears in \cite[(2.8)]{ElAlaouiSellke2022}. For step order parameters, $X$ observed at the breakpoints of $\gamma$ and at time $1$ is a Markov chain with explicit tilted Gaussian transitions (\Cref{I:gap:diffusion}). The variable $X_1-\kappa\ge0$ plays the role of the gap of a typical constraint. \section{Main results}\label{I:sec:results} Our first result is the Parisi formula for smooth activations. We consider two classes of activations: \begin{enumerate}[label=\textup{(\Alph*)}] \item\label{I:class-A} $U\in C^2(\R)$, $\sup U<\infty$, and $\norm{U'}_\infty+\norm{U''}_\infty<\infty$; \item\label{I:class-B} $U\in C^2(\R)$ is concave, $\sup U<\infty$, and $\norm{U''}_\infty<\infty$. \end{enumerate} No sign condition is imposed on $U'$ in class~\ref{I:class-A}, and the activations in class~\ref{I:class-B} may tend to $-\infty$ quadratically. \begin{theorem}\label{I:thm:parisi} Let $\alpha>0$ and let $U$ belong to class~\ref{I:class-A} or class~\ref{I:class-B}. Then \[ \lim_{N\to\infty}F_N(U)=\cP_*(U), \qquad f_N(U)\longrightarrow\cP_*(U)\quad\text{in probability}. \] Moreover, the infimum defining $\cP_*(U)$ is attained, and every minimizer $\gamma$ satisfies $Q_\gamma\le r_U$, where $r_U<1$ is an explicit number depending only on $\alpha$ and $U$, defined in \eqref{I:pp:rU}. If $U$ belongs to class~\ref{I:class-A}, then $r_U\le\alpha\norm{U'}_\infty^2/(1+\alpha\norm{U'}_\infty^2)$. \end{theorem} Class~\ref{I:class-A} contains activations that are neither monotone nor concave. This generality is used for the gaps: adding a bounded source term to a smooth approximation of the hard wall destroys monotonicity and concavity. Class~\ref{I:class-B} contains smooth concave penalties comparable to $-\beta(\ell-x)_+^2$, which we use to pass to hard constraints. The bound on $Q_\gamma$ says that the minimizers for a smooth activation stay a fixed distance away from overlap one. For the hard wall this fails as the margin approaches the critical margin; see \Cref{I:thm:jamming}(iv). For $\alpha>2$ the critical margin is negative, and the feasible sets $S_N(\kappa)$ with $\kappa<0$ are not convex. Our second result identifies their exponential volume and the largest achievable margin. \begin{theorem}\label{I:thm:capacity} Let $\alpha>2$. Then $-\infty<\kc(\alpha)<0$, and the following statements hold. \begin{enumerate}[label=\textup{(\roman*)}] \item For every $\kappa<\kc(\alpha)$, $N^{-1}\log V_N(\kappa)\to\cP_*(\kappa)$ in probability. \item $\kappa_N\to\kc(\alpha)$ in probability. \item For every $\kappa\ge\kc(\alpha)$ and every $C<\infty$, $\Prob\{N^{-1}\log V_N(\kappa)>-C\}\to0$. \end{enumerate} \end{theorem} Part (iii) states that the normalized log volume tends to $-\infty$ at the critical margin itself; it does not assert that $S_N(\kc)$ is empty. The assumption $\alpha>2$ enters only through the inequality $\kc(\alpha)<0$; for every $\alpha>0$, part (i) holds at the margins $\kappa<\min\{\kc(\alpha),0\}$ (see the end of \Cref{I:vol:proof}). For $\alpha>2$, parts (i) and (iii) confirm the free-energy part of a conjecture of El Alaoui and Sellke \cite[Conjecture~1]{ElAlaouiSellke2022}, and part (ii) confirms its capacity part in the form $\kappa_N\to\kc(\alpha)$. The proofs of \Cref{I:thm:capacity} and of the gap theorems below rest on the following properties of the variational problem, which hold for every $\alpha>0$. For $\kappa\in\R$ put $C_\kappa=\frac12\bigl(\kappa+\sqrt{\kappa^2+4}\bigr)$, and let $C_0=-\log\Phi(-1)$. \begin{theorem}\label{I:thm:jamming} Let $\alpha>0$. \begin{enumerate}[label=\textup{(\roman*)}] \item If $\cP_*(\kappa)>-\infty$, then the infimum defining $\cP_*(\kappa)$ is attained, every minimizer $\gamma$ satisfies $1-Q_\gamma>\e^{2\cP_*(\kappa)}$, and the set of minimizers is compact in $L^1([0,1])$. \item $-\infty<\kc(\alpha)\le\kRS(\alpha)<\infty$. If $\alpha>2$, then $\kc(\alpha)<0$. \item $\cP_*$ is finite, nonincreasing, concave, and continuous on $(-\infty,\kc)$, and $\cP_*(\kappa)=-\infty$ for $\kappa\ge\kc$. More precisely, with $C_*=C_{\kc}$, for all sufficiently small $\eta>0$, \[ \cP_*(\kc-\eta)\le\frac12\log\eta+\frac12\log(2\alpha C_*)+\frac12 . \] \item If $\cP_*(\kappa)>-\infty$ and $\gamma$ is a minimizer, then \[ \cP_*(\kappa)\ge\frac{1+\alpha}2\log(1-Q_\gamma)-\alpha\bigl(C_0+\log(1+C_\kappa)\bigr). \] Consequently there are $C,\eta_0>0$, depending only on $\alpha$, such that for $0<\eta\le\eta_0$ every minimizer $\gamma$ of $\cP_{\kc-\eta}$ satisfies \[ 1-Q_\gamma\le C\eta^{1/(1+\alpha)},\qquad \lambda_\gamma(0)\le C\eta^{1/(2+2\alpha)} . \] \item There is $\kappa_0(\alpha)<\kc(\alpha)$ such that, for every $\kappa\le\kappa_0(\alpha)$, every minimizer of $\cP_\kappa$ equals $\1_{[q,1]}$ for some $q\in(0,1)$. In particular $\cP_*(\kappa)=\min_{q\in[0,1)}\PRS_\kappa(q)$. If $\alpha\le1$, the same holds for every $\kappa<\kc(\alpha)$. \end{enumerate} \end{theorem} Part (iii) says that the jamming transition is continuous on the exponential scale: the Parisi value tends to $-\infty$ as the margin increases to $\kc$, and it does so at least logarithmically fast. Part (iv) shows that a very negative value pins the top of the support of every minimizer close to one. Together, (iii) and (iv) give the collapse of minimizers at the critical margin: their tops tend to one and their masses tend to zero, at power rates. This collapse is the input for the gap theorems. By part (v) and \Cref{I:thm:capacity}(i), for $\alpha>2$ and $\kappa\le\kappa_0(\alpha)$, \[ \frac1N\log V_N(\kappa)\longrightarrow\min_{q\in[0,1)}\PRS_\kappa(q) \quad\text{in probability}, \] which is Gardner's replica-symmetric formula for the free energy \cite{Gardner1988}. Thus Gardner's formula holds at sufficiently negative margins; the margin $\kappa_0(\alpha)$ is not explicit. \Cref{I:thm:jamming} is proved in \Cref{I:hw:proof-jamming}, and part (v) is \Cref{I:hw:rsdeep}. Two structural facts behind it hold for every order parameter: an exact value identity (\Cref{I:fun:value}) and the universal slope bound $0<\lambda_\gamma(0)\,\partial_xu_\gamma(0,0;\kappa)0$ and $c>0$ such that, for $00$ sufficiently small and every $\kappa<\kRS(\alpha)$ the function $\PRS_\kappa$ of \eqref{I:eq-RS} has a unique minimizer $q^*(\kappa)$ on $[0,1)$, and $q^*(\kappa)\in(0,1)$. By Part~II, \Cref{II:prop:crossing}, there is also a numerical constant $c_2>0$ such that exactly one margin $\kappa\in(\kRS(\alpha)-c_2,\kRS(\alpha))$ satisfies \[ \alpha\E V(Z^*)^2=1,\qquad Z^*=\frac{\sqrt{q^*}\,G-\kappa}{\sqrt{1-q^*}},\quad q^*=q^*(\kappa). \] This margin is the de Almeida--Thouless margin $\kd(\alpha)$. The displayed condition says that the replicon eigenvalue of the replica-symmetric solution vanishes (see \cite{Bouten1994,FPSUZ2017}); on $(\kd,\kRS)$ this eigenvalue is negative, and the replica-symmetric solution is unstable (Part~II, \Cref{II:prop:crossing}). The ordering of thresholds (Part~II, \Cref{II:thm:thresholds}) states that there are numerical constants $e_0,c,C>0$ such that, for $00$ such that \[ \Prob\Bigl\{\frac1N\log V_N(\kappa)\le\min_{q\in[0,1)}\PRS_\kappa(q)-\eps\Bigr\} \longrightarrow1 . \] \end{enumerate} \end{corollary} Part (i) sharpens \Cref{I:cor:gardner} to exact order: the largest achievable margin converges to a limit that lies below Gardner's prediction by an amount of exact order $(\alpha-2)^3$, and above the de Almeida--Thouless margin by an amount of order $(\alpha-2)^2$. The thermodynamic limit is taken at fixed $e$, and part (i) describes the subsequent behavior of the limit $\kc(2+e)$ as $e\downarrow0$. Part (ii) shows that Gardner's formula for the free energy fails on the whole interval $(\kd(\alpha),\kRS(\alpha))$. By \Cref{I:thm:jamming}(v) it holds at sufficiently negative margins, and by \Cref{I:thm:capacity}(i) and Part~II, \Cref{II:thm:main}(i), it holds at the margins in $[\kd(\alpha)-c'e^2,\kd(\alpha)]$ for a numerical constant $c'>0$. The corollary follows directly from the results above. Part (i) is \Cref{I:thm:capacity}(ii) together with \eqref{I:eq-II-thresholds}, since $\kc-\kd=(\kRS-\kd)-(\kRS-\kc)$. For part (ii), the minimum of $\PRS_\kappa$ is attained at $q^*(\kappa)$, so it is finite. Let $\kappa\in(\kd,\kc)$. By \Cref{I:thm:jamming}(i),(iii), minimizers of $\cP_\kappa$ exist, and by Part~II, \Cref{II:cor:nonrs}, none of them is replica symmetric. Hence $\cP_*(\kappa)<\PRS_\kappa(q^*(\kappa))$, and \Cref{I:thm:capacity}(i) gives the claim. For $\kappa\in[\kc,\kRS)$ the claim follows from \Cref{I:thm:capacity}(iii). Part~II also proves a lower bound on the Parisi value that matches \Cref{I:thm:jamming}(iii) up to the constant in front of the logarithm, through a comparison with a zero-temperature functional (Part~II, \Cref{II:fun:rates}): for every $\alpha>2$ there are $C,\eta_0>0$, depending only on $\alpha$, such that \begin{equation}\label{I:eq-II-rates} \alpha\log\eta-C\le\cP_*(\kc-\eta)\le\frac12\log\eta+C \qquad(0<\eta\le\eta_0). \end{equation} Together with \Cref{I:thm:capacity}(i), this logarithmic divergence gives the following. \begin{corollary}\label{I:cor:logvolume} Let $\alpha>2$. There are $C,\eta_0>0$, depending only on $\alpha$, such that for every $0<\eta\le\eta_0$, \[ \Prob\Bigl\{\alpha\log\eta-C\le\frac1N\log V_N(\kc-\eta) \le\frac12\log\eta+C\Bigr\}\longrightarrow1 . \] \end{corollary} Equivalently, with probability tending to one, $\e^{-CN}\eta^{\alpha N}\le V_N(\kc-\eta)\le\e^{CN}\eta^{N/2}$. On the exponential scale the feasible volume thus vanishes polynomially in the distance $\eta$ to the critical margin. The upper bound uses only \Cref{I:thm:capacity,I:thm:jamming}, and we do not claim that either exponent is optimal. We next describe the configurations that nearly achieve the maximal margin, through their empirical gap measures $\nu_{N,\sigma}$. \begin{theorem}\label{I:thm:gaps} Let $\alpha>2$. There is a deterministic measure $\nu_\alpha$ on $[0,\infty)$ with $\nu_\alpha([0,\infty))=\alpha$ and $\nu_\alpha(\{0\})=1$, whose positive part satisfies \begin{equation}\label{I:gap:small-gaps} \nu_\alpha((0,t])\le\frac{\alpha\log3}{\log(\sqrt{2\pi}/t)}\qquad(00$ there is $\rho>0$ such that \[ \Prob\Bigl\{\sup_{\sigma\in S_N:\ m_N(\sigma)\ge\kc-\rho} \Bigl|\int\psi\dd\nu_{N,\sigma}-\int\psi\dd\nu_\alpha\Bigr|\le\eps\Bigr\} \longrightarrow1. \] \item For every $c<(\alpha-2)^2/(2\alpha)$ and all sufficiently large $N$, with probability at least $1-\e^{-cN}$ every maximizer $\sigma_*$ of $m_N$ has exactly $N$ contacts. For any choice of maximizers, $\nu_{N,\sigma_*}\to\nu_\alpha$ weakly in probability, and \[ \frac1N\#\{a\le M:00$ of $\nu_\alpha$. \item For every choice of margins $k\uparrow\kc(\alpha)$ and minimizers $\gamma_k$ of $\cP_k$, the measures $\alpha\Law(X^{\gamma_k,k}_1-k)$ converge weakly to $\nu_\alpha$. \end{enumerate} \end{theorem} Here $\nu_{N,\sigma_*}\to\nu_\alpha$ weakly in probability means that $\int\psi\dd\nu_{N,\sigma_*}\to\int\psi\dd\nu_\alpha$ in probability for every bounded continuous $\psi$, uniformly over the maximizers. The count $N$ in part (ii) is isostatic in the sense recalled above: it equals the $N-1$ degrees of freedom on the sphere plus one for the optimized margin. The identity $\nu_\alpha(\{0\})=1$ and the bound \eqref{I:gap:small-gaps} show that, in the limit, no additional macroscopic set of constraints has small positive gaps: the mass $\nu_\alpha((0,t])$ of the small positive gaps tends to zero as $t\downarrow0$, at least as fast as a constant times $1/\log(1/t)$. Nearly optimal configurations need not have exactly $N$ contacts, but by part (i) their gap statistics are close to $\nu_\alpha$, and by part (iii) the limiting gap law is identified through Parisi minimizers approaching the critical margin. The theorem does not identify the behavior of $\nu_\alpha$ near zero, where physics predicts a power law. The last theorem identifies the gap statistics of typical feasible configurations at a fixed margin below the critical one. \begin{theorem}\label{I:thm:gibbs-gaps} Let $\alpha>2$ and $\kappa<\kc(\alpha)$, let $\gamma$ be a minimizer of $\cP_\kappa$, and let $\psi:[0,\infty)\to\R$ be bounded and $L$-Lipschitz with $L>0$. For every $\eps>0$, with probability tending to one, $V_N(\kappa)>0$ and \begin{multline}\label{I:gap:gibbs-concentration} \mu_N\Bigl(\Bigl\{\sigma\in S_N(\kappa):\Bigl|\frac1N\sum_{a=1}^M\psi(b_a\cdot\sigma-\kappa) -\alpha\E\psi(X^{\gamma,\kappa}_1-\kappa)\Bigr|>\eps\Bigr\}\Bigr)\\ \le\exp\Bigl\{-\frac{\eps^2N}{8\alpha L^2\lambda_\gamma(0)}\Bigr\}\,V_N(\kappa). \end{multline} Consequently, writing $\langle\cdot\rangle_{N,\kappa}$ for expectation under the uniform probability measure on $S_N(\kappa)$, \begin{equation}\label{I:gap:gibbs-average} \Bigl\langle\frac1N\sum_{a=1}^M\psi(b_a\cdot\sigma-\kappa)\Bigr\rangle_{N,\kappa} \longrightarrow\alpha\E\psi(X^{\gamma,\kappa}_1-\kappa)\qquad\text{in probability}, \end{equation} and all minimizers of $\cP_\kappa$ induce the same law of $X^{\gamma,\kappa}_1$. \end{theorem} Applied to finitely many test functions at once, the theorem shows that all but an exponentially small fraction of the feasible set has the gap statistics of the Parisi diffusion. The rate in \eqref{I:gap:gibbs-concentration} depends on the minimizer only through its mass $\lambda_\gamma(0)$, and any minimizer may be used. We do not prove that the minimizers of $\cP_\kappa$ are unique, and none of our arguments requires it; the last assertion of the theorem shows that they all induce the same terminal law. For mixed $p$-spin models, uniqueness follows from strict convexity of the Parisi functional \cite{AuffingerChen2015Uniqueness,JagannathTobasco2016}, which is not known for $\cP_\kappa$. For generic mixed $p$-spin models, Auffinger and Jagannath expressed spin statistics through solutions of partial differential equations \cite{AuffingerJagannath2019}. Throughout, $C$ and $c$ denote positive constants. A numerical constant depends on nothing; otherwise the dependence of a constant is indicated in the statement where it appears, as in ``depending only on $\alpha$''. Their values may change from one occurrence to the next. Limits in probability refer to the joint law of the patterns and of any auxiliary randomness introduced in the proofs. \section{Ideas of the proof}\label{I:sec:ideas} The proof has three parts. Steps~1 and~2 below prove the Parisi formula for smooth activations. Steps~3 and~4 analyze the deterministic variational problem for the hard wall. Steps~5 and~6 transfer these results to the hard constraints of the random model and to the gaps. \Cref{I:fig:map} shows how the steps depend on one another. \begin{figure}[htbp] \centering \input{figures/part1-map} \caption{Selected dependencies among the steps of the proof. Each solid box names a step or chapter and, where there is one, the main result it proves; an arrow runs from a box to a box that uses it. The dashed boxes are the corollaries that also use results of Part~II. \Cref{I:cor:gardner} combines \Cref{I:thm:gardner,I:thm:capacity} and is not drawn, and neither are the analytic properties of the Parisi functional collected in \Cref{I:sec:functional}, which are used throughout. Among them, the lemmas of \Cref{I:fun:finite-cutoff} are derived from Part~II, \Crefrange{II:fun:sec:gauss}{II:fun:sec:kkt} and \Cref{II:app:functional}, which do not use Part~I.} \label{I:fig:map} \end{figure} \paragraph{Step 1: The upper bound.} The argument follows the proof of \cite[Theorem~4.1]{Mourrat2021}, a step in Mourrat's upper bound for bipartite models; the opening of \Cref{I:sec:upper} compares our argument with his. We use a Guerra-type interpolation \cite{Guerra2003} between the perceptron and a decoupled system on a Ruelle probability cascade \cite{Ruelle1987,BolthausenSznitman1998}. Since $\cP_U$ is continuous in the order parameter, it suffices to treat order parameters $\gamma_h$ with finitely many atoms $h_0\le\cdots\le h_k$ of equal mass. As the interpolation time $t$ runs from $0$ to $1$, the argument of each activation moves from a Gaussian field on the leaves of the cascade, with covariance $h_l$ between two leaves that branch at level $l$, to $b_a\cdot\sigma$. The coordinates are coupled to a second cascade field, whose levels $p=(p_0,\dots,p_k)$ are free parameters. The derivative in $t$ of the interpolating free energy $u_N$ is expressed through two overlaps: the spin overlap $R_{12}=\sigma^1\cdot\sigma^2/N$, and the \emph{response overlap} $X_{12}=M^{-1}\sum_aU'(S^1_a)U'(S^2_a)$, where $S^l_a$ is the interpolated field of constraint $a$ for replica $l$. Overlaps of the derivatives of the activation enter in a similar way in Talagrand's treatment of the perceptron \cite[Chapters~2 and~3]{TalagrandVolI}, \cite[Chapter~8]{TalagrandVolII}. Let $J$ be the level at which the leaves of two replicas branch, $w_l$ the probability that $J=l$, and $r_l$ and $\zeta_l$ the conditional means of $R_{12}$ and of $\alpha_NX_{12}$ given $J=l$, where $\alpha_N=M/N$. Then \[ \partial_tu_N=-\frac12\sum_{l=0}^kw_l\Bigl[(r_l-h_l)\zeta_l +\Cov\bigl(R_{12},\alpha_NX_{12}\bigm|J=l\bigr)\Bigr]. \] Neither term has a sign. We control the products through the choice of $p$. At a maximizer over $p$ of $u_N+\frac12\sum_lw_lh_lp_l$, the first-order conditions, together with the monotonicity $0\le\zeta_0\le\cdots\le\zeta_k$, make the total contribution of the products nonpositive. This monotonicity holds for every activation in class~\ref{I:class-A}; we derive it from an exact change of measure on a cascade with marks. For Gaussian fields the corresponding monotonicity is \cite[Proposition~14.3.2]{TalagrandVolII} and \cite[Lemma~2.4]{Mourrat2021}. A maximum principle in $t$ alone then bounds the free energy by the maximum over $p$ at $t=0$, up to the covariance terms. To control these, small Gaussian perturbations of the Hamiltonian, of the form used in \cite[Section~4]{Mourrat2021}, enforce approximate Ghirlanda--Guerra identities \cite{GhirlandaGuerra1998} for the pair of overlap arrays at the points where the maximum principle is applied. Mourrat's quantitative synchronization \cite{Mourrat2021}, which builds on Panchenko's synchronization mechanism \cite{Panchenko2015,Panchenko2018Vector}, then makes the conditional variance of $R_{12}$ small, and the Cauchy--Schwarz inequality makes the covariances small whatever the sign of $U'$. This is why no sign condition on $U'$ is needed. At $t=0$ the system splits into the row term $\alpha_Nu_{\gamma_h}(0,0;U)$ and a spherical term. An exact computation with Gaussian cascades bounds the spherical term plus $\frac12\sum_lw_lh_lp_l$, maximized over $p$, by $\CS(\gamma_h)+o(1)$; a computation of this kind for spherical spin glasses is in \cite{Talagrand2006Spherical}. This is carried out in \Cref{I:sec:upper}. \paragraph{Step 2: The lower bound.} We use the Aizenman--Sims--Starr scheme \cite{AizenmanSimsStarr2003} in the spherical form of \cite{Chen2013}. For multi-species spherical models, Bates and Sohn combined W.-K.\ Chen's cavity computation with Panchenko's synchronization \cite{Panchenko2015,Panchenko2018Vector} to prove the lower bound \cite{BatesSohn2022EJP}. We follow the same route, with the response overlap in the role of the overlap of the second species. The scheme bounds the free energy from below by the increment of the expected log partition function when $n$ coordinates, and about $\alpha n$ constraints, are added to a system with $N$ coordinates. We write the sphere of dimension $N+n$ exactly as a product of the sphere of dimension $N$ and a density for the $n$ new coordinates (compare \cite{DiaconisFreedman1987}). The increment then splits into a row term and a coordinate term. In the limit the row term is at least $\alpha n\,u_\zeta(0,0;U)$, where $\zeta$ is the limiting overlap law. After a Gaussian interpolation, the coordinate term becomes a Gaussian integral over the new coordinates $\varepsilon\in\R^n$. After a perturbation (see \cite{PanchenkoBook} and \cite[(24)--(28)]{Panchenko2015}), the joint law of the spin overlap and the response overlap satisfies the Ghirlanda--Guerra identities. By synchronization and the characterization of such arrays \cite{Panchenko2013Ultrametricity,Panchenko2015,Panchenko2018Vector,Mourrat2021,DominguezMourrat2024}, it is a limit of Ruelle probability cascades with paired nondecreasing levels. On such a cascade the coordinate term is an explicit Gaussian integral, which for general paired levels is not the entropy of any order parameter. Two identities of the system determine the result. First, a Gaussian integration by parts in the rows expresses the mean effective precision of the new coordinates through the two overlaps. In the limit, this \emph{precision identity} shows that the smallest precision is at least one, so that the Gaussian integral is finite, and it cancels the constant terms. Second, the law of the patterns is invariant under rotations. Under a uniformly random rotation, the overlap $\varepsilon^1\cdot\varepsilon^2/n$ of the new coordinates of two replicas is close to their full overlap, with mean squared error at most $3/n$ in the limit. Further, $\E\norm\varepsilon^2/n=1$, and this norm identity turns the coordinate term into $n\CS(\gamma)$, where $\gamma$ is the law of the mean overlap of the new coordinates at the branching level, and the overlap estimate makes $\gamma$ close to $\zeta$. Hence, as $N\to\infty$, the increment per added coordinate is at least \[ \cP_U(\gamma)-O(n^{-1/2})\ge\cP_*(U)-O(n^{-1/2}). \] Rotation invariance thus forces the cavity term to equal the Crisanti--Sommers entropy, and no minimizer or critical point of a cavity functional has to be identified. The main technical issue is the unbounded Gaussian integral over $\varepsilon$. We compute it on balls, pass to the limit $N\to\infty$ there, evaluate the limit on finite cascades, and remove the cutoff on these cascades, where the precision identity gives uniform control. This is carried out in \Cref{I:sec:lower}. \Cref{I:sec:parisi} combines the two bounds. There a truncation argument, which moves the mass of an order parameter above the level $r_U$ down to $r_U$ and so decreases $\cP_U$, gives the attainment of the infimum and the bound $Q_\gamma\le r_U$ for minimizers. \paragraph{Step 3: The variational problem at hard margins.} For the hard wall we prove an exact \emph{value identity}, valid for every order parameter: \[ \cP_\kappa(\gamma)=\frac12\log\delta+\alpha\E h(Z)-\frac\delta2D(Q) -\frac12\int_0^Q\gamma(q)D(q)\dd q . \] Here $Q=Q_\gamma$, $\delta=1-Q$, $Z=(X_Q-\kappa)/\sqrt\delta$ is the normalized top position of the Parisi diffusion, $h(z)=\log\Phi(z)+\frac12R(z)^2<0$, and $D(q)=\alpha\E\,\partial_xu_\gamma(q,X_q;\kappa)^2-\int_0^q\lambda_\gamma^{-2}$ is twice the density of the first variation of $\cP_\kappa$. El Alaoui and Sellke computed this first variation at a minimizer whose support is an interval, in directions supported on that interval \cite[Proposition~16]{ElAlaouiSellke2022} (see the opening of \Cref{I:sec:functional}). Here it is used at every order parameter and in every direction that keeps the top of the support below a fixed level; \Cref{I:sec:hardwall} recalls the first-order conditions known for mixed $p$-spin models. At a minimizer over the order parameters whose top is at most a fixed level $r<1$, the last two terms are nonpositive, so $\cP_\kappa(\gamma)<\frac12\log\delta$. A finite value therefore keeps the top of every minimizer away from one, which gives existence and compactness of minimizers. The second tool is the \emph{universal slope bound} \[ 0<\lambda_\gamma(0)\,\partial_xu_\gamma(0,0;\kappa)\kc$, where the unconstrained value is $-\infty$. The value identity gives $1-Q_\gamma>\e^{2w}$, so its slope in the margin is at most $\alpha C_k\e^{-2w}$. Concavity and the limit $k\downarrow\kc$ give $\cP_*(\kc-\eta)\le w+\alpha\eta C_{\kc}\e^{-2w}$, and the choice $w=\frac12\log(2\alpha C_{\kc}\eta)$ gives \Cref{I:thm:jamming}(iii). For a minimizer over all order parameters, the value identity holds without the last two terms. Since $h(z)$ is bounded below by $-\log(1+z_-)$ up to an additive constant, and the slope bound controls the negative part of $Z$, Jensen's inequality gives the pinning bound of \Cref{I:thm:jamming}(iv). Combined with (iii) at the margin $\kc-\eta$, it forces $1-Q_\gamma\le C\eta^{1/(1+\alpha)}$. Finally, weak top stability, a lower bound on the quantity $S(Q)=\alpha\delta^2\E\,\partial_{xx}u_\gamma(Q,X_Q;\kappa)^2$ for constrained minimizers with more than one support point, gives $\lambda_\gamma(0)\le C(1-Q_\gamma)^{1/2}$. At very negative margins the same lower bound cannot hold, so every minimizer is replica symmetric; this gives $\kc>-\infty$ and \Cref{I:thm:jamming}(v). No zero-temperature functional is needed. This is carried out in \Cref{I:sec:hardwall}. \paragraph{Step 4: The critical margin near $\alpha=2$.} To go below Gardner's prediction we evaluate $\cP_\kappa$ at an explicit order parameter with two support points near one, \[ \gamma_\delta=a\1_{[p,Q)}+\1_{[Q,1]},\qquad Q=1-\delta,\quad p=Q(1-\ell),\quad a=\frac\delta{Q\ell}, \] which splits the atom of a replica-symmetric order parameter and places a thin layer of relative width $\ell$ below the top. The homogeneity and the monotonicity of the Cole--Hopf step and a Gaussian tail bound give $\delta\,\cP_\kappa(\gamma_\delta)\le Q\,J_\ell(\kappa/\sqrt Q)+\frac12\delta\log\delta$ for an explicit Gaussian integral $J_\ell$, and \[ J_\ell(k)=\frac{1-\alpha\E(k-G)_+^2}4-\frac\ell2\Bigl(\log2-\frac12\Bigr) \bigl(\alpha\Phi(k)-1\bigr)+O(\ell^{3/2}). \] If $J_\ell(k)<0$, then $\cP_\kappa(\gamma_\delta)\to-\infty$ as $\delta\downarrow0$ for every $\kappa>k$, so that $\kc\le k$. At $k=\kRS$ the first term vanishes, while $\alpha\Phi(\kRS)-1$ is of exact order $e=\alpha-2$. The layer therefore gains an amount of order $e\ell$ at a cost of order $\ell^{3/2}$, and the choice $\ell\asymp e^2$ makes $J_\ell(\kRS)$ negative, of order $e^3$. Lowering the margin by a small multiple of $e^3$ keeps it negative. The argument uses no minimizer and no regularity theory beyond the recursion for this single order parameter. Unlike the bounds of Stojnic and of Montanari, Zhong, and Zhou recalled at the beginning of this chapter, the bound comes from the Parisi functional at an explicit order parameter, and the thin layer of width $\ell\asymp e^2$ gives the order $e^3$ near $\alpha=2$. An explicit trial order parameter was used in the same way by Toninelli to show that the Sherrington--Kirkpatrick model is not replica symmetric beyond the de Almeida--Thouless line \cite{Toninelli2002}. This is carried out in \Cref{I:sec:gardner}. \paragraph{Step 5: From soft free energy to hard feasible volume.} The upper bounds on the volume and on $\kappa_N$ come from Step~1, applied to smooth approximations of the hard wall from above, together with a lemma showing that a single feasible configuration forces an exponentially nonnegligible feasible volume after a fixed decrease of the margin. For the lower bound, the Parisi formula for a concave penalty comparable to $-\beta(\ell-x)_+^2$ bounds from below the volume of the configurations whose total squared violation $\sum_a(k_0-b_a\cdot\sigma)_+^2$, at a level $k_0$ slightly below $\ell$, is at most $tN$, for small $t>0$. Two geometric facts turn this into feasible volume. First, a small violation can be repaired by a small displacement. Using the smallest singular value of the pattern matrix over sparse sets of rows (see \cite{Vershynin2012}), a separation argument moves such a configuration by at most $4\sqrt{tN}$ to a point that satisfies every constraint at a slightly smaller margin. The opening of \Cref{I:sec:hard-volume} compares this repair with that of El Alaoui and Sellke \cite[Lemma~11]{ElAlaouiSellke2022}. Second, the repair map may collapse volume, so we do not use its image. Instead we add to the configuration a Gaussian vector tangent to the sphere, with variance $\rho^2$ in each direction, and project back to $S_N$. This preserves $\mu_N$. Translating the Gaussian vector by a displacement $d$ lowers the probability of a symmetric set by at most the factor $\e^{-\norm d^2/(2\rho^2)}$, and \v{S}id\'ak's inequality \cite{Sidak1967} shows that, with probability at least $\e^{-N\epsilon}$ for a small $\epsilon>0$, the Gaussian vector moves no constraint by more than a small amount. Hence the perturbed point is feasible at a slightly smaller margin with probability at least $\e^{-N\epsilon'}$, where $\epsilon'$ is small when $t$ is small compared with $\rho^2$. The upper bounds are proved in \Cref{I:up:hard-sec,I:up:cap-sec} and the lower bound in \Cref{I:sec:hard-volume}. \paragraph{Step 6: Gap statistics.} Gap averages enter through the free energy of the feasible set with a source term $\beta\sum_a\psi(b_a\cdot\sigma-k)$, for a bounded Lipschitz test function $\psi$. Adding the source to a smooth approximation of the hard wall destroys monotonicity, and this is where the generality of class~\ref{I:class-A} is used. We prove a \emph{quadratic source estimate}: the Parisi functional at $\gamma$ with the source exceeds its linearization in $\beta$, whose slope is $\alpha\E\psi(X^{\gamma,k}_1-k)$, by at most $\frac12\alpha\Lip(\psi)^2\beta^2\lambda_\gamma(0)$. For a step order parameter the effect of the source is an iterated logarithmic moment along the Parisi diffusion observed at the breakpoints of $\gamma$. The transitions of this Markov chain are Gaussian laws tilted by log-concave weights. Like Gaussian laws, they have sub-Gaussian linear statistics and map Lipschitz functions to Lipschitz functions with the same constant (for log-concave tilts of Gaussian laws see \cite{BrascampLieb1976,Caffarelli2000}), and a backward induction with a second-order expansion at each step gives the estimate. Combined with Jensen's inequality and the upper bound of Step~1, it yields one deviation inequality. With probability tending to one, simultaneously for every set $T\subseteq S_N(k)$ of positive volume, the average over $T$ of $N^{-1}\sum_a\psi(b_a\cdot\sigma-k)$ differs from $\alpha\E\psi(X^{\gamma,k}_1-k)$ by at most \[ \frac{\cP_k(\gamma)-N^{-1}\log\mu_N(T)}\beta +\frac{\alpha\Lip(\psi)^2\beta}2\,\lambda_\gamma(0)+\eps . \] For \Cref{I:thm:gibbs-gaps} we apply it at a fixed margin $\kappa<\kc$ to the set of feasible configurations whose gap average deviates by more than $\eps$, and \Cref{I:thm:capacity}(i) shows that this set is an exponentially small fraction of $S_N(\kappa)$. For \Cref{I:thm:gaps}, a single configuration of nearly maximal margin is surrounded by a set of configurations that are feasible at a margin smaller by $\eta$, have almost the same gaps, and have volume at least $\e^{-N(\log(1/\eta)+C)}$. Optimizing over $\beta$ leaves an error of order $\eta+\{\lambda_\gamma(0)(1+\log(1/\eta))\}^{1/2}$, which tends to zero by the collapse of minimizers in \Cref{I:thm:jamming}(iv). The exact count of contacts does not use the variational problem. By Wendel's theorem \cite{Wendel1962}, $\kappa_N<0$ with probability exponentially close to one when $\alpha>2$. The maximizers of $m_N$ then correspond to the points of the polytope $\{y\in\R^N:g_a\cdot y\ge-1\text{ for all }a\}$ farthest from the origin. These are vertices, and Gaussian general position gives exactly $N$ contacts. A union bound over sets of $N$ rows gives a finite-$N$ form of \eqref{I:gap:small-gaps} (\Cref{I:gap:contacts}). This is carried out in \Cref{I:sec:gaps}. \medskip \Cref{I:sec:functional} collects the analytic properties of the Parisi functional used throughout: the Cole--Hopf recursion for step order parameters, Lipschitz continuity in the order parameter, and, for the hard wall, a priori bounds, signs of derivatives, and the first variation. \Cref{I:thm:parisi} is proved in \Cref{I:sec:parisi}, \Cref{I:thm:jamming} in \Cref{I:sec:hardwall}, \Cref{I:thm:gardner} in \Cref{I:sec:gardner}, \Cref{I:thm:capacity} in \Cref{I:sec:hard-volume}, and \Cref{I:thm:gibbs-gaps,I:thm:gaps} in \Cref{I:sec:gaps}. The corollaries follow from these theorems and, for \Cref{I:cor:thresholds,I:cor:logvolume}, from Part~II, as explained next to their statements. The lemmas of \Cref{I:fun:finite-cutoff}, parts (i)--(iv) of \Cref{I:hw:gauss}, one Gaussian bound in the proof of \Cref{I:hw:deep} and the expansions of \Cref{I:sec:gardner} are derived from Part~II, \Crefrange{II:fun:sec:gauss}{II:fun:sec:kkt}, \Cref{II:lem:krs} and \Cref{II:app:functional}, which do not use Part~I. Through them, \Cref{I:thm:capacity,I:thm:jamming,I:thm:gardner,I:thm:gaps,I:thm:gibbs-gaps} also rest on Part~II; \Cref{I:thm:parisi} does not. \chapter{The Parisi functional}\label{I:sec:functional} This chapter collects the deterministic facts about the Parisi functional that are used in the rest of this part. \Cref{I:pf:step} introduces the Cole--Hopf step $\cT_{m,s}$ and the backward recursion that solves the Parisi equation \eqref{I:eq-pde} when the order parameter is a step function. It records the monotonicity of the recursion in its terminal datum, which is used in \Cref{I:sec:upper,I:sec:parisi,I:sec:hard-volume,I:sec:gaps}, and the homogeneity of the Cole--Hopf step, which is used in \Cref{I:sec:gardner}. It also shows that neither the Crisanti--Sommers entropy nor the hard-wall value depends on the cutoff used to define it. \Cref{I:pf:smooth} proves that, for activations in class~\ref{I:class-A} or class~\ref{I:class-B}, the value $u_\gamma(0,0;U)$ is Lipschitz in $\gamma$ for the $L^1$ distance, with an explicit constant (\Cref{I:pf:trial-continuity,I:lo-phi-cont}). This defines $u_\gamma$ for general order parameters, and it lets the upper bound (\Cref{I:sec:upper}), the lower bound (\Cref{I:sec:lower}), the truncation argument of \Cref{I:sec:parisi} and the passage to hard constraints in \Cref{I:sec:hard-volume} move between step order parameters and general ones. These continuity estimates rest on one identity, \eqref{I:pf:FKdiff} in \Cref{I:pf:diff}, which writes the difference of two solutions with the same terminal datum as an integral along a diffusion. Its only input is a formula for the derivatives of one Cole--Hopf step (\Cref{I:pf:onestep}). \Cref{I:fun:finite-cutoff} treats the hard-wall functional $\cP_\kappa$ on order parameters that equal one above a fixed level $r<1$. There we state a priori bounds on the solution and on the associated diffusion (\Cref{I:pf:apriori}), Lipschitz continuity in the order parameter (\Cref{I:pf:coef-cont}), signs and a martingale (\Cref{I:pf:analytic}), and the first variation (\Cref{I:pf:first-variation}). These are the inputs for the analysis of the variational problem in \Cref{I:sec:hardwall} and for the gap statistics in \Cref{I:sec:gaps}. Most of them are derived from Part~II, as explained at the beginning of \Cref{I:fun:finite-cutoff}. The hard-wall functional $\cP_\kappa$ was studied by El Alaoui and Sellke in \cite[Section~4]{ElAlaouiSellke2022}. For step order parameters they solved the Parisi equation with the terminal datum \eqref{I:eq-hard-terminal} by the Cole--Hopf recursion \cite[(4.16)]{ElAlaouiSellke2022} and proved a priori bounds on the solution \cite[Proposition~12]{ElAlaouiSellke2022}. They proved Lipschitz continuity in the order parameter \cite[Proposition~13]{ElAlaouiSellke2022}, extended the solution to general order parameters \cite[(4.12)]{ElAlaouiSellke2022}, and proved that it has one-sided derivatives in time \cite[Lemma~14]{ElAlaouiSellke2022}. They also computed the first variation of $\cP_\kappa$ in terms of the Parisi diffusion \cite[(2.8), (4.13), and (4.15)]{ElAlaouiSellke2022}, at a minimizer whose support is an interval, in directions supported on that interval, and deduced the first-order conditions $D=0$ and $S=1$ on that interval \cite[Proposition~16]{ElAlaouiSellke2022}, with $D$ and $S$ as in \eqref{I:fun:D}. Their arguments build on work on the Sherrington--Kirkpatrick and mixed $p$-spin models: Guerra's Lipschitz bound in the order parameter \cite{Guerra2001}, the Feynman--Kac proof of it by Jagannath and Tobasco \cite[Lemma~14 and Remark~15]{JagannathTobasco2016}, the diffusion and the regularity estimates of Auffinger and W.-K.\ Chen \cite{AuffingerChen2015Uniqueness,AuffingerChenProperties}, and the first-variation formulas of W.-K.\ Chen \cite{Chen2017Variational} and of El Alaoui, Montanari, and Sellke \cite[Proposition~6.8]{AlaouiMontanariSellke2021}. This chapter and Part~II, Chapter~2 follow the same route, with the following differences. Concavity of the solution is obtained from Pr\'ekopa's theorem (\Cref{I:pf:onestep}) rather than from an explicit computation along the recursion, which uses that the order parameter is nondecreasing \cite[proof of (4.6)]{ElAlaouiSellke2022}. The bound $-\partial_{xx}u_\gamma\le(1-\bar q)^{-1}$ of \cite[(4.6)]{ElAlaouiSellke2022} is sharpened to the strict bound $-\partial_{xx}u_\gamma<1/\lambda_\gamma$ (\Cref{I:pf:analytic}). The constants in \Cref{I:fun:finite-cutoff} are uniform for $\kappa$ in a compact interval. The first variation is computed at every order parameter in the class $\cU_r$ of order parameters equal to one on $[r,1]$, in every direction that stays in $\cU_r$ (\Cref{I:pf:first-variation}), and not only at a minimizer. Finally, the Lipschitz estimates of \Cref{I:pf:smooth} cover activations in class~\ref{I:class-B}, which are concave and may tend to $-\infty$ quadratically. Throughout this chapter, $G$ denotes a standard Gaussian variable and $G'$ an independent copy of it, and $\phi$ and $\Phi$ denote the standard Gaussian density and distribution function. All functions are Borel measurable, and $\norm{\cdot}_{L^1}$ is the norm of $L^1([0,1])$. \section{Order parameters and the Cole--Hopf recursion}\label{I:pf:step} Recall from \Cref{I:sec:model} that an \emph{order parameter} is a nondecreasing, right-continuous function $\gamma:[0,1]\to[0,1]$ that is identically one on $[\bar q,1]$ for some $\bar q<1$, and that $\cU$ is the set of order parameters. For $\gamma\in\cU$ we write \[ Q_\gamma=\inf\{q\in[0,1]:\gamma(q)=1\}<1, \qquad \lambda_\gamma(q)=\int_q^1\gamma(t)\dd t . \] Thus $\gamma$ is the distribution function of a probability measure on $[0,Q_\gamma]$. More generally, we identify every probability measure on $[0,1]$ with its distribution function, a nondecreasing right-continuous function $\gamma:[0,1]\to[0,1]$ with $\gamma(1)=1$. \paragraph{The Cole--Hopf step.} For $m\ge0$, $s\ge0$, and a function $f:\R\to[-\infty,\infty)$ that is bounded above, define \begin{equation}\label{I:pf:Top} \cT_{m,s}f(x)= \begin{cases} \dfrac1m\log\E\e^{mf(x+\sqrt s\,G)}, & m>0,\\[2mm] \E f(x+\sqrt s\,G), & m=0, \end{cases} \end{equation} with the conventions $\e^{-\infty}=0$ and $\log0=-\infty$. The expectations exist because $f$ is bounded above, $\cT_{m,s}f$ takes values in $[-\infty,\sup f]$, and $\cT_{m,0}f=f$. For functions $f,g$ that are bounded above, $a\in\R$, $m\ge0$, and $s,s'\ge0$, the following hold. \begin{enumerate}[label=\textup{(T\arabic*)}] \item\label{I:pf:T1} If $f\le g+a$ on $\R$, then $\cT_{m,s}f\le\cT_{m,s}g+a$ on $\R$. \item\label{I:pf:T2} $\cT_{m,s}\cT_{m,s'}f=\cT_{m,s+s'}f$. \item\label{I:pf:T3} $t\,\cT_{m,s}f=\cT_{m/t,s}(tf)$ for every $t>0$. \item\label{I:pf:T4} $\cT_{m,s}f\ge\cT_{0,s}f$. \end{enumerate} If $f\equiv-\infty$, then $\cT_{m,s}f\equiv-\infty$ for all $m,s$, and \ref{I:pf:T2}--\ref{I:pf:T4} for $f$ are immediate; below we assume $f\not\equiv-\infty$, so that $\sup f$ is finite. Property \ref{I:pf:T1} holds because $y\mapsto\e^{my}$ is nondecreasing and $\cT_{m,s}(g+a)=\cT_{m,s}g+a$. For \ref{I:pf:T2}, write $x+\sqrt{s+s'}\,G$ as $x+\sqrt s\,G+\sqrt{s'}\,G'$. If $m>0$, Tonelli's theorem applied to $\e^{mf}\ge0$ gives \[ \E\exp\bigl\{m\,\cT_{m,s'}f(x+\sqrt s\,G)\bigr\} =\E\,\e^{mf(x+\sqrt s\,G+\sqrt{s'}\,G')} =\E\,\e^{mf(x+\sqrt{s+s'}\,G)}, \] and if $m=0$ the same argument applies to $\sup f-f\ge0$. Property \ref{I:pf:T3} follows from \eqref{I:pf:Top}, and \ref{I:pf:T4} is Jensen's inequality $\E\e^{mf}\ge\e^{m\E f}$ for $m>0$ and an equality for $m=0$. The operator $\cT_{m,s}$ solves the Parisi equation over an interval on which the order parameter equals a constant $m$; this is the Cole--Hopf transformation. Indeed, if $u$ solves \eqref{I:eq-pde} with $\gamma=m>0$, then $h=\e^{mu}$ satisfies \[ \partial_qh+\frac12\partial_{xx}h =mh\Big(\partial_qu+\frac12\partial_{xx}u+\frac m2(\partial_xu)^2\Big)=0, \] the backward heat equation, whose solutions are Gaussian averages of their later values. When $m=0$ the equation is itself the backward heat equation. The recursion below turns this computation into a definition. \paragraph{The step recursion.} Let $T\in[0,1]$. A \emph{step function} on $[0,T)$ is a function $\gamma=\sum_{j=0}^nm_j\1_{[t_j,t_{j+1})}$ with $0=t_00$ and $s>0$, \[ \e^{mg(s,x)}=\int_\R\frac{\e^{-(y-x)^2/(2s)}}{\sqrt{2\pi s}}\, \e^{mf(y)}\dd y ,\qquad 0<\e^{mf}\le\e^{mC'} . \] Differentiation under the integral shows that the right side is positive, $C^\infty$ in $(s,x)$, and solves $\partial_sh=\frac12\partial_{xx}h$; reading the Cole--Hopf computation backwards gives the equation for $g$. For $m=0$ the same argument applies to $g$ itself, since $\abs{f(y)}\le C'(1+y^2)$. For the limit, fix $\rho>0$. For $m>0$, \[ \abs{\E\e^{mf(x+\sqrt s\,G)}-\e^{mf(x)}} \le\sup_{\abs{y-x}\le\rho}\abs{\e^{mf(y)}-\e^{mf(x)}} +2\e^{mC'}\Prob\{\sqrt s\abs G>\rho\}, \] and for $m=0$ the last term is replaced by $\E[\abs{f(x+\sqrt s\,G)-f(x)};\sqrt s\abs G>\rho]$, which is at most $C''(1+x^2)\Prob\{\sqrt s\abs G>\rho\}^{1/2}$ for $s\le1$, with $C''$ depending only on $C'$, by the Cauchy--Schwarz inequality. Letting $s\downarrow0$ and then $\rho\downarrow0$, the continuity of $f$ shows that $\cT_{m,s}f\to f$ locally uniformly as $s\downarrow0$ (for $m>0$ because $\e^{mf}$ is continuous and positive), and hence $g(s,x)\to f(x_0)$. \paragraph{General order parameters.} For $\gamma\in\cU$ choose $\bar q\in[Q_\gamma,1)$. There are step order parameters $\gamma_n$ with $\gamma_n=1$ on $[\bar q,1]$ and $\norm{\gamma_n-\gamma}_{L^1}\le1/n$. Indeed, let $0=t_0<\cdots0$, and put $g=\cT_{m,s}f$. For $x\in\R$ let $P_x$ be the law of $Y=x+\sqrt s\,G$ tilted by $\e^{mf(Y)}$, that is, $\dd P_x/\dd\Law(Y)=\e^{mf(Y)}/\E\e^{mf(Y)}$. Then $g\in C^2(\R)$, $g\le\sup f$, and \begin{equation}\label{I:pf:onestep-formula} g'(x)=\E_{P_x}f'(Y),\qquad g''(x)=\E_{P_x}f''(Y)+m\Var_{P_x}\big(f'(Y)\big). \end{equation} Consequently the following hold. \begin{enumerate}[label=\textup{(\alph*)}] \item If $\abs{f'}\le L$ on $\R$, then $\abs{g'}\le L$ and $\abs{g''}\le\sup\abs{f''}+mL^2$. \item If $f$ is concave and $f''\ge-K$ on $\R$, then $g$ is concave and $g''\ge-K$. \end{enumerate} \end{lemma} \begin{proof} Since $f'$ has at most linear growth, $f''$ is bounded, and $\e^{mf}\le\e^{m\sup f}$, the function $\e^{mf}$ and its first two derivatives $mf'\e^{mf}$ and $(mf''+m^2f'^2)\e^{mf}$, evaluated at $x+\sqrt s\,G$, are dominated by $C(1+\abs G)^2$ for $x$ in a compact set, where $C$ depends on the compact set. Hence we may differentiate $\E\e^{mf(x+\sqrt s\,G)}$ twice under the expectation, and dividing by $\E\e^{mf(Y)}$ gives \eqref{I:pf:onestep-formula} for $m>0$. For $m=0$ we have $P_x=\Law(Y)$, and the same argument applied to $\E f(x+\sqrt s\,G)$ gives $g'=\E f'(Y)$ and $g''=\E f''(Y)$. Part (a) follows from \eqref{I:pf:onestep-formula} and $\Var_{P_x}f'(Y)\le L^2$. In part (b), $g''\ge-K$ by \eqref{I:pf:onestep-formula}. If $m=0$, then $g$ is an average of concave functions. If $m>0$, put $h(x,y)=\exp\{mf(y)-(y-x)^2/(2s)\}$. Its logarithm is concave in $(x,y)$, and $\int_\R h(x,y)\dd y=\sqrt{2\pi s}\,\e^{mg(x)}$. By Pr\'ekopa's theorem \cite[Theorem~6]{Prekopa1973}, the marginal of a log-concave function on $\R^2$ is log-concave, so $mg$ is concave. \end{proof} For the hard-wall terminal datum, the formula for $g''$ in \eqref{I:pf:onestep-formula} appears in \cite[(4.17)]{ElAlaouiSellke2022}. Part~II, \Cref{II:appA:rec} proves the same formulas for concave nondecreasing $f$; here part~(a) also covers data that are neither concave nor monotone, as in class~\ref{I:class-A}. As in Part~II, \Cref{II:appA:rec}, part~(b) gives concavity one step at a time. Hence \Cref{I:pf:diff}(i) holds for all step functions with values in $[0,1]$, whereas Part~II, \Cref{II:appA:diff} and \cite[proof of (4.6)]{ElAlaouiSellke2022} assume the coefficient nondecreasing. The next lemma compares the solutions for two step coefficients with the same terminal datum, at a general terminal time $T$. Part~II, \Cref{II:appA:diff} proves the identity \eqref{I:pf:FKdiff} for a class of concave nondecreasing terminal data that contains the hard-wall datum. \begin{lemma}\label{I:pf:diff} Let $T\in(0,1]$ and let $F\in C^2(\R)$ be bounded above with $F''$ bounded. Let $\gamma,\nu:[0,T)\to[0,1]$ be step functions, let $u_\gamma=u_\gamma(\cdot,\cdot\,;F)$ and $u_\nu=u_\nu(\cdot,\cdot\,;F)$ be given by \eqref{I:pf:recursion} with terminal datum $F$ at time $T$, and put $w=u_\gamma-u_\nu$. Assume one of the following. \begin{enumerate}[label=\textup{(\Alph*)}] \item $\abs{F'}\le L$ on $\R$. \item $F$ is concave and $F''\ge-K$ on $\R$ for a constant $K\ge0$. In this case put $A=\sup F-F(0)+\abs{F'(0)}+K$. \end{enumerate} Then the following hold for all $(q,x)\in[0,T]\times\R$. \begin{enumerate}[label=\textup{(\roman*)}] \item In case (A), $u_\gamma(q,\cdot)\in C^2(\R)$ and $\abs{\partial_xu_\gamma}\le L$. In case (B), $u_\gamma(q,\cdot)\in C^2(\R)$ and \begin{gather*} -K\le\partial_{xx}u_\gamma\le0,\qquad F(0)-\abs{F'(0)}\abs x-\frac K2(1+x^2)\le u_\gamma(q,x)\le\sup F,\\ \abs{\partial_xu_\gamma(q,x)}\le A+K\abs x . \end{gather*} The same holds for $u_\nu$. \item Let $B$ be a standard Brownian motion and let $Y$ solve $\dd Y_s=\beta(s,Y_s)\dd s+\dd B_s$ on $[q,T]$ with $Y_q=x$, where $\beta=\frac\gamma2(\partial_xu_\gamma+\partial_xu_\nu)$. This equation has a unique strong solution, and \begin{equation}\label{I:pf:FKdiff} w(q,x)=\frac12\int_q^T(\gamma-\nu)(s)\, \E\big[(\partial_xu_\nu)^2(s,Y_s)\big]\dd s . \end{equation} \item In case (A), $\abs{w(q,x)}\le\frac{L^2}2\int_q^T\abs{\gamma-\nu}\dd s$. In case (B), \[ \abs{w(q,x)}\le C_{A,K}(1+x^2)\int_q^T\abs{\gamma-\nu}\dd s, \qquad C_{A,K}=A^2+3K^2\e^{2K}(16+A^2). \] \end{enumerate} \end{lemma} \begin{proof} Refining the partitions, we may assume that $\gamma$ and $\nu$ are constant on the intervals $[t_j,t_{j+1})$ of a common partition $0=t_0<\cdotsq$, put $t_j'=\max\{q,t_j\}$, and let $t_j'0$, \[ \frac{t\phi(t)}{1+t^2}<\Phi(-t)<\frac{\phi(t)}t, \qquad\text{equivalently}\qquad t0$ and $00,\qquad 0<-f_r''(x)=\frac{V(z)}{1-r}<\frac1{1-r}. \end{equation} Every derivative of $f_r$ has polynomial growth, uniformly for $\kappa$ in a compact interval; this gives the case $r=0$ of \Cref{I:pf:apriori}(v). Indeed, $R'=-zR-R^2$, so every derivative of $R$ is a polynomial in $z$ and $R$, and $00$ for $q\in[0,Q]$, and $u$ is defined on $[0,1)\times\R$ by \eqref{I:pf:hardonflat}. We write \[ \cL=\partial_q+\frac12\partial_{xx}+\gamma b\,\partial_x \] for the generator of the diffusion \begin{equation}\label{I:pf:X} \dd X_q=\gamma(q)\,b(q,X_q)\dd q+\dd B_q,\qquad X_0=0, \end{equation} where $B$ is a standard Brownian motion. For $\gamma\in\cU_r$ we consider $X$ on $[0,r]$, and we write $X^{s,x}$ for the solution of the equation in \eqref{I:pf:X} on $[s,r]$ with $X^{s,x}_s=x$; it exists and is unique by \Cref{I:pf:apriori}(iii) for step order parameters and by \Cref{I:pf:coef-cont} in general. On $[0,Q]$, $X$ is the Parisi diffusion \eqref{I:eq-diffusion} of \Cref{I:sec:model}, and the processes defined with two admissible values of $r$ agree on their common interval. It is also the diffusion of Part~II, \eqref{II:fun:X}. When two order parameters $\gamma,\widetilde\gamma$ are compared, we indicate the order parameter as a subscript, as in $u_{\widetilde\gamma}$, $b_\gamma$, or $X^{\widetilde\gamma}$. With this notation, the hard-wall functional \eqref{I:eq-kappac} with cutoff $\bar q=Q$ reads \begin{equation}\label{I:fun:def} \cP_\kappa(\gamma)=\alpha u(0,0)+\frac12\int_0^Q\frac{\dd q}{\lambda(q)} +\frac12\log\delta =\alpha u(0,0)+\frac12\int_0^1\Big(\frac1{\lambda(q)}-\frac1{1-q}\Big)\dd q, \end{equation} the second form by \eqref{I:pf:CScutofffree}, and $\cP_*(\kappa)=\inf_{\gamma\in\cU}\cP_\kappa(\gamma)$. \paragraph{A priori bounds and continuity.} The next lemma collects the bounds on the solution and the diffusion that justify It\^o's formula and the Feynman--Kac formula in the rest of this part. \begin{lemma}\label{I:pf:apriori} Fix $r\in[0,1)$ and a compact interval $I$. There is a constant $C$, and for each $p\ge1$ and $k\ge1$ there are constants $C_p$ and $C'_k$, depending only on $r$, $I$, and $p$ or $k$, such that for every $\kappa\in I$ and every step order parameter $\gamma\in\cU_r$ the following hold on $[0,r]\times\R$. \begin{enumerate}[label=\textup{(\roman*)}] \item $-C(1+x^2)\le u(q,x)\le0$. \item On each open interval where $\gamma$ is constant, $u$ is smooth and \[ \cL b=0,\qquad \cL c=\gamma c^2 . \] \item $0\le c\le(1-r)^{-1}$ and $\abs{b(q,x)}\le C(1+\abs x)$. In particular the drift $\gamma b$ in \eqref{I:pf:X} is Lipschitz in $x$ with constant $(1-r)^{-1}$ and has linear growth, so the equation in \eqref{I:pf:X} has a unique strong solution $X^{s,x}$ from every starting point $(s,x)\in[0,r]\times\R$. \item $\E\sup_{t\in[s,r]}\abs{X^{s,x}_t}^p\le C_p(1+\abs x)^p$; in particular $\E\sup_{q\le r}\abs{X_q}^p\le C_p$. \item $\abs{\partial_x^ku(q,x)}\le C'_k(1+\abs x)^{C'_k}$, and $\partial_x^ku$ is continuous on $[0,r]\times\R$. \end{enumerate} \end{lemma} \begin{proof} For $r=0$ the statements concern $u=f_0$ only and follow from \eqref{I:pf:terminal} and the polynomial growth of the derivatives of $f_0$ noted after it; let $r>0$. Every statement is contained in Part~II, Chapter~2, applied with $K_0=\max_{\kappa\in I}\abs\kappa$; the constants there depend only on $r$, $K_0$, and $p$ or $k$. Part~(i) and the bound on $b$ in~(iii) are Part~II, \Cref{II:lem:regularity}(b). By Part~II, \Cref{II:lem:regularity}(c), $00$, \[ \sup\Big\{\abs{\partial_x^ku_{\widetilde\gamma}(q,x)-\partial_x^ku_\gamma(q,x)}: q\in[0,r],\ \abs x\le A_0\Big\}\longrightarrow0 \qquad\text{as }\eps\to0, \] uniformly in $\kappa\in I$ and in the step order parameters. When $X^{\widetilde\gamma}$ and $X^\gamma$ start from the same point $(s,x)\in[0,r]\times\R$ and are driven by the same Brownian motion, \begin{equation}\label{I:pf:Xbound} \E\sup_{t\in[s,r]}\abs{X^{\widetilde\gamma}_t-X^\gamma_t}^p \le C_p\eps^p(1+\abs x)^p . \end{equation} Consequently the solution $u$, its spatial derivatives, and the diffusion $X$ extend from step order parameters to every $\gamma\in\cU_r$ by $L^1$ approximation within $\cU_r$, and the bounds of \Cref{I:pf:apriori}(i), (iii)--(v) and of this lemma hold for all $\gamma,\widetilde\gamma\in\cU_r$. If $\gamma\in\cU_r$ equals a constant $m$ on an interval $[a,a')\subset [0,r]$, then $u(q,\cdot)=\cT_{m,a'-q}u(a',\cdot)$ for $q\in[a,a')$, and $u$ is smooth and satisfies $\cL b=0$ and $\cL c=\gamma c^2$ on $(a,a')\times\R$. Finally, \begin{equation}\label{I:pf:Plip} \abs{\cP_\kappa(\widetilde\gamma)-\cP_\kappa(\gamma)} \le\Big(\alpha C+\frac r{2(1-r)^2}\Big)\norm{\widetilde\gamma-\gamma}_{L^1} \qquad(\gamma,\widetilde\gamma\in\cU_r,\ \kappa\in I). \end{equation} In particular $\cP_\kappa$ is continuous on the compact set $\cU_r$, uniformly for $\kappa\in I$, and attains its minimum over $\cU_r$. \end{lemma} \begin{proof} For $r=0$ the class $\cU_0$ has one element; let $r>0$. We apply the results of Part~II, Chapter~2 with $K_0=\max_{\kappa\in I}\abs\kappa$; they hold for all $\gamma,\widetilde\gamma\in\cU_r$, and their constants depend only on $r$, $K_0$, and $p$ or $k$. The solution for general $\gamma\in\cU_r$ is the limit of the solutions for step approximations in $\cU_r$ (Part~II, \Cref{II:def:orderparam}(iii)). By Part~II, \Cref{II:lem:regularity}(f), for every $k\ge0$, \[ \abs{\partial_x^ku_{\widetilde\gamma}(q,x)-\partial_x^ku_\gamma(q,x)} \le C_k\eps(1+\abs x^{p_k})\qquad\text{on }[0,r]\times\R, \] with $p_0=2$ and $p_1=1$. The cases $k=0,1$ give \eqref{I:pf:wbounds}, the cases $k\ge2$ give the convergence of the higher derivatives, and the derivatives of $u_\gamma$ are the limits of those of its step approximations. The bound \eqref{I:pf:Xbound} is Part~II, \Cref{II:fun:diffusion}(b); in particular the diffusions of step approximations converge to $X^\gamma$. Part~II, \Cref{II:lem:regularity}(a)--(c) and \Cref{II:fun:diffusion}(a) hold for general $\gamma\in\cU_r$, so the proof of \Cref{I:pf:apriori} gives its parts~(i) and (iii)--(v) for all $\gamma\in\cU_r$. If $\gamma=m$ on $[a,a')$, then $u(q,\cdot)=\cT_{m,a'-q}u(a',\cdot)$ for $q\in[a,a')$ by Part~II, \Cref{II:fun:colehopf}(a), and $u$ is smooth and satisfies $\cL b=0$ and $\cL c=\gamma c^2$ on $(a,a')\times\R$ by Part~II, \Cref{II:lem:regularity}(a) and \Cref{II:lem:generator}(a). Finally, the first bound in \eqref{I:pf:wbounds} at $(0,0)$ controls the energy term of $\cP_\kappa$, and \eqref{I:pf:CSlip} controls the entropy term; this proves \eqref{I:pf:Plip}. Compactness of $\cU_r$ was shown in \Cref{I:pf:step}. \end{proof} We record a consequence of \Cref{I:pf:coef-cont}. If $\gamma\in\cU_r\subset\cU_{r'}$ with $r0,\qquad 00$ and take $I=\{\kappa\}$. \proofpart{Step order parameters.} Let $\gamma$ be a step order parameter and fix $(s,x)\in[0,r]\times\R$. By It\^o's formula on each open interval of constancy and $\cL b=0$, $\dd b(t,X^{s,x}_t)=\partial_xb(t,X^{s,x}_t)\dd B_t=-c(t,X^{s,x}_t)\dd B_t$ there. Since $b$ is continuous on $[0,r]\times\R$ with linear growth and $0\le c\le K$ by \Cref{I:pf:apriori}(iii),(v), the identity extends to the endpoints of the intervals as in Step 3 of the proof of \Cref{I:pf:diff}, and \[ b(t,X^{s,x}_t)=b(s,x)-\int_s^tc(t',X^{s,x}_{t'})\dd B_{t'}\qquad(t\in[s,r]). \] With $(s,x)=(0,0)$ this is \eqref{I:pf:bmart}, and since $0\le c\le K$ the stochastic integral is a square-integrable martingale on $[0,r]$. \proofpart{General order parameters.} Let $\gamma\in\cU_r$, let $\gamma_n\in\cU_r$ be step approximations, and write $X^n=X^{\gamma_n}$, $b_n=b_{\gamma_n}$, and $c_n=c_{\gamma_n}$. By \Cref{I:pf:coef-cont}, $c_n\to c$ uniformly on compact subsets of $[0,r]\times\R$, and $X^n\to X$ in every moment, uniformly on $[0,r]$. By \eqref{I:pf:wbounds} and the Lipschitz bound $\abs{\partial_xb}\le K$, \[ \abs{b_n(q,X^n_q)-b(q,X_q)}\le C\eps_n(1+\abs{X^n_q})+K\abs{X^n_q-X_q}, \] with $\eps_n=\norm{\gamma_n-\gamma}_{L^1}$, so $b_n(q,X^n_q)\to b(q,X_q)$ in $L^2$. Moreover $c_n(s,X^n_s)\to c(s,X_s)$ in probability for each $s$, by the locally uniform convergence of $c_n$, the continuity of $c$, and the convergence of $X^n$; since $0\le c_n,c\le K$, dominated convergence and It\^o's isometry give $\int_0^qc_n(s,X^n_s)\dd B_s\to\int_0^qc(s,X_s)\dd B_s$ in $L^2$. The identity \eqref{I:pf:bmart} for $\gamma_n$ therefore passes to the limit, almost surely for each fixed $q\in[0,r]$. Both sides of \eqref{I:pf:bmart} have continuous paths, since $b$ and $X$ are continuous and the stochastic integral has a continuous version, so the identity holds simultaneously for all rational $q$ on one event of probability one, and then for all $q\in[0,r]$ by continuity. \end{proof} The martingale identity \eqref{I:pf:bmart} is the hard-wall form of \cite[Lemma~2]{AuffingerChen2015Uniqueness}. Part~II, \Cref{II:lem:generator}(b) states the martingale property of $b(q,X_q)$ without the stochastic integral. \paragraph{The first variation.} For $\gamma\in\cU_r$ and $q\in[0,r]$ set \begin{equation}\label{I:fun:D} D(q)=\alpha\E b(q,X_q)^2-\int_0^q\lambda(t)^{-2}\dd t, \qquad S(q)=\alpha\lambda(q)^2\E c(q,X_q)^2 . \end{equation} The first term of $D$ comes from the energy term of $\cP_\kappa$ and the second from the entropy term, as the proof of Part~II, \Cref{II:prop:kkt}(a) shows. Since $v=\lambda c$, these are the functions $D$ and $S$ of Part~II, \eqref{II:fun:scalars}. The function $D$ and the first variation below are the basic tools of \Cref{I:sec:hardwall}. \begin{lemma}\label{I:pf:first-variation} Let $r\in[0,1)$, $\kappa\in\R$, and $\gamma,\widetilde\gamma\in\cU_r$. The function $D$ is continuously differentiable on $[0,r]$ with \[ D'=\frac{S-1}{\lambda^2}. \] Moreover, with $\gamma_\theta=\gamma+\theta(\widetilde\gamma-\gamma)\in\cU_r$ for $\theta\in[0,1]$, \begin{equation}\label{I:fun:variation} \left.\frac{\dd}{\dd\theta}\cP_\kappa(\gamma_\theta)\right|_{\theta=0+} =\frac12\int_0^r(\widetilde\gamma-\gamma)(q)\,D(q)\dd q . \end{equation} \end{lemma} \begin{proof} The formula for $D'$ is among the generator identities of Part~II, \Cref{II:lem:generator}(c)(i), and \eqref{I:fun:variation} is the first equality in Part~II, \eqref{II:fun:variation} (\Cref{II:prop:kkt}(a)); both hold for all $\gamma,\widetilde\gamma\in\cU_r$ and $\kappa\in\R$. \end{proof} Formula \eqref{I:fun:variation} is the hard-wall analogue of the directional derivative of the Parisi functional of mixed $p$-spin models computed by W.-K.\ Chen \cite[Theorem~2]{Chen2017Variational}; see also \cite[Proposition~6.8]{AlaouiMontanariSellke2021}. The first variation of El Alaoui and Sellke \cite[(4.15)]{ElAlaouiSellke2022} involves the same function $D$; the beginning of this chapter compares the two. The formula for $D'$ is the analogue of \cite[Proposition~3]{AuffingerChenProperties}, where Auffinger and W.-K.\ Chen differentiate the corresponding function for mixed $p$-spin models. Talagrand computed the derivative of the Parisi functional when the order parameter is transported by a map close to the identity \cite{TalagrandParisiMeasures}. \chapter{The upper bound}\label{I:sec:upper} This chapter proves that $\cP_U(\gamma)$ bounds the free energy from above for every order parameter $\gamma$ and every activation $U$ in class~\ref{I:class-A} (\Cref{I:up-soft}), and deduces upper bounds for the feasible volume and for the maximal margin. \Cref{I:up-cascade} is a change of measure on a Ruelle probability cascade with marks, which turns Gibbs averages over the leaves into expectations along a Markov chain, and \Cref{I:up-cmconsequences} deduces that conditional overlaps are nondecreasing in the branching level. \Cref{I:gc:finite} computes a Gaussian cascade exactly. \Cref{I:up-derivatives} gives the derivatives of the interpolating free energy. \Cref{I:up-concentration} proves concentration of the free energy and computes its derivatives in the perturbation parameters, and \Cref{I:up-contact,I:up-sync} use it to control the conditional covariance of the two overlaps. \Cref{I:up-sphere} bounds the spherical term by the Crisanti--Sommers entropy. \Cref{I:up-comparisonprop} combines these facts through a maximum principle in the interpolation time. \Cref{I:up-hardvolume} passes to the hard wall, with a bounded source term, and \Cref{I:up-cap,I:up-margin}, with \Cref{I:up-sidak}, bound the maximal margin. Step~1 of \Cref{I:sec:ideas} outlines the argument and introduces the notation used below. The proof follows that of \cite[Theorem~4.1]{Mourrat2021}, a step in Mourrat's proof that the Hamilton--Jacobi limit bounds the free energy of bipartite models from above \cite[Theorem~1.1]{Mourrat2021}. As there, the atoms of the order parameter have equal mass, the derivative in $t$ is a sum of products of conditional overlap means and a conditional covariance (see \cite[(2.23)]{Mourrat2021}), and Mourrat's perturbation enforces the Ghirlanda--Guerra identities only at contact points (\Cref{I:up:interpolation,I:up:contact-sec}). Four things differ. The row side carries a non-Gaussian activation $U$, with no sign condition on $U'$, in place of Mourrat's bilinear Gaussian energy. Only the spin side has free cascade levels, so the viscosity argument becomes a maximum principle in $t$ for the maximum over $p$ (\Cref{I:up:comparison}). The monotonicity of the response overlaps $\zeta_l$ follows from a change of measure on a marked cascade; its Gaussian analogue is the monotonicity of the normalized derivatives of the enriched free energy in the cascade parameters \cite[Lemma~2.4]{Mourrat2021} (see also \cite[Proposition~14.3.2]{TalagrandVolII}). The bound closes on the Parisi functional $\cP_U(\gamma_h)$ instead of the solution of a Hamilton--Jacobi equation. Throughout this chapter, $b_a=g_a/\sqrt N$ for the rows $g_a$ of $\bG$, $\alpha_N=M/N\to\alpha$, and $\alpha'=\sup_N\alpha_N<\infty$. For $U$ in class~\ref{I:class-A} we write $U_{\max}=\sup U$ and $L=\norm{U'}_\infty$. \begin{theorem}\label{I:up-soft} Let $U$ belong to class~\ref{I:class-A}. For every $\gamma\in\cU$, \begin{equation}\label{I:up-softbound} \limsup_{N\to\infty}F_N(U)\le\cP_U(\gamma). \end{equation} Moreover, $\Prob\{f_N(U)>\cP_U(\gamma)+\varepsilon\}\to0$ for every $\varepsilon>0$. \end{theorem} \Cref{I:sec:parisi} combines \Cref{I:up-soft} with the lower bound of \Cref{I:sec:lower}. \section{A change of measure on a marked cascade}\label{I:up:cascade-sec} The interpolation places Gaussian fields on the nodes of a Ruelle probability cascade. Its free energy is an iterated logarithmic moment over the levels of the tree, and to differentiate it we need to express averages over the cascade weights as expectations along a Markov chain on the marks. The following change-of-measure identity does this. It also gives the law of the branching level of two leaves sampled from the weights, and it shows that conditional overlaps are nondecreasing in the level. \paragraph{The cascade.} Fix $k\ge0$ and $0<\theta_1<\cdots<\theta_k<1$, and set $\theta_0=0$, $\theta_{k+1}=1$, and $w_l=\theta_{l+1}-\theta_l$ for $0\le l\le k$. The nodes of depth $l$ are the strings $\beta\in\N_*^l$, with the empty string as the root, and the leaves are the nodes of depth $k$. For a leaf $\beta$ we write $\beta|l$ for its ancestor at depth $l$, and for two leaves we put $\beta\wedge\beta'=\max\{l:\beta|l=\beta'|l\}$. At every node of depth $l-1$, independently, place a Poisson point process on $(0,\infty)$ with intensity measure $\theta_lx^{-1-\theta_l}\dd x$. Its points, in decreasing order, are attached to the edges from that node to its children; $\pi_\beta$ denotes the point attached to the edge into the node $\beta$. For a leaf $\beta$ put \[ \Pi_\beta=\prod_{l=1}^k\pi_{\beta|l},\qquad \Pi=\sum_\beta\Pi_\beta,\qquad v_\beta=\frac{\Pi_\beta}{\Pi}. \] The weights $(v_\beta)$ are the Ruelle probability cascade \cite{Ruelle1987}. For $k=0$ the root is the only leaf, and $\Pi=v=1$. Independently of the cascade, attach to each node $\beta$ of depth $l\ge1$ a random \emph{mark} $e_\beta$ with law $P_l$ on a standard Borel space; marks at different nodes are independent. The root mark $e_0$ is fixed for now; below it contains the disorder shared by all leaves. For a leaf $\beta$ write $e_{[\beta]}=(e_0,e_{\beta|1},\ldots,e_{\beta|k})$ for the marks along its path. Let $X_k(e_0,\ldots,e_k)$ be a measurable function of a path of marks, and define backwards, for $l=k,\ldots,1$, \begin{equation}\label{I:up-recursion} X_{l-1}(e_0,\ldots,e_{l-1})=\frac1{\theta_l}\log\int \e^{\theta_lX_l}\dd P_l(e_l), \qquad \mathsf K_l(\dd e_l\mid e_0,\ldots,e_{l-1}) =\e^{\theta_l(X_l-X_{l-1})}P_l(\dd e_l). \end{equation} We assume that every $X_l$ is finite on every path. Then each $\mathsf K_l$ is a probability kernel. We write $\E_{\mathsf K}$ for expectation over the Markov chain $(e_0,e_1,\ldots,e_k)$ started from $e_0$ with the kernels $\mathsf K_1,\ldots,\mathsf K_k$, and, for $0\le l\le k$, $\E_{\mathsf K,l}$ for expectation over two paths $(e,e')$ that coincide up to depth $l$, follow the chain up to depth $l$, and then continue independently with the kernels $\mathsf K_{l+1},\ldots,\mathsf K_k$. \begin{lemma}\label{I:up-cascade} The quantities $\Pi$ and $W=\sum_\beta v_\beta \e^{X_k(e_{[\beta]})}$ are positive and finite almost surely. Let $\omega_\beta=v_\beta \e^{X_k(e_{[\beta]})}/W$ be the associated Gibbs weights on the leaves. Then, for every measurable $\varphi$ that is nonnegative, or integrable under the law on the right side, \begin{align} \E\sum_\beta\omega_\beta\,\varphi(e_{[\beta]})&=\E_{\mathsf K}\varphi(e),\label{I:up-onepath}\\ \E\sum_{\beta,\beta'}\omega_\beta\omega_{\beta'}\1_{\{\beta\wedge\beta'=l\}} \varphi(e_{[\beta]},e_{[\beta']})&=w_l\,\E_{\mathsf K,l}\varphi(e,e'), \qquad 0\le l\le k.\label{I:up-twopath} \end{align} Moreover, conditionally on every value of the root mark $e_0$, \begin{equation}\label{I:up-stableidentity} \log W=X_0(e_0)+\log\Pi'-\log\Pi, \qquad \Pi'\overset d=\Pi,\qquad \E\abs{\log\Pi}^2<\infty . \end{equation} \end{lemma} In words, \eqref{I:up-onepath} says that the path of marks of a leaf sampled from the Gibbs weights is distributed as the chain with the kernels $\mathsf K_l$, and \eqref{I:up-twopath} says that two independently sampled leaves branch at depth $l$ with probability $w_l$, whatever the function $X_k$, and that their paths then share the first $l$ marks and continue independently. These are known properties of Ruelle probability cascades, and they rest on the invariance property of Bolthausen and Sznitman \cite{BolthausenSznitman1998}; see \cite[Chapter~2]{PanchenkoBook}. With marks and the kernels $\mathsf K_l$, \eqref{I:up-onepath} and \eqref{I:up-twopath} correspond to \cite[Theorems~6 and~7]{PanchenkoTalagrand2007Cascades}, and \eqref{I:up-stableidentity} to \cite[(3.12)--(3.13)]{PanchenkoTalagrand2007Cascades}; Panchenko and Talagrand use them to write Guerra's interpolation on a cascade. See also \cite{Arguin2007}. Steps~1 and~3 of the proof follow \cite[Lemmas~2 and~3]{PanchenkoTalagrand2007Cascades}, and Step~4 follows Mourrat's derivation of the law of $\beta\wedge\beta'$ by Gaussian integration by parts \cite{Mourrat2022CJM}, which adapts the derivation of \cite[(2.82)]{PanchenkoBook}. \begin{proof} For $k=0$ we have $W=\e^{X_0(e_0)}$, $\omega=1$, and $w_0=1$, and all assertions hold with $\Pi=\Pi'=1$. Let $k\ge1$. We divide the proof into four steps. \proofstep{1}{A Poisson computation.} Let $(x_n)$ be a Poisson process with intensity $\theta x^{-1-\theta}\dd x$, let $(e_n)$ be independent marks with law $P$, and let $f$ be measurable with $c=\int \e^{\theta f}\dd P<\infty$. By the mapping and marking theorems for Poisson processes \cite[Chapters~2 and~5]{Kingman1993}, the pairs $(x_n\e^{f(e_n)},e_n)$ form a Poisson process with intensity \[ c\,\theta y^{-1-\theta}\dd y\otimes c^{-1}\e^{\theta f(e)}P(\dd e). \] Indeed, for a test function $\chi$ the substitution $y=x\e^{f(e)}$ gives \[ \iint\chi(x\e^{f(e)},e)\,\theta x^{-1-\theta}\dd x\,P(\dd e) =\iint\chi(y,e)\,\theta y^{-1-\theta}\e^{\theta f(e)}\dd y\,P(\dd e). \] After the first coordinate is multiplied by $c^{-1/\theta}$, the first factor becomes $\theta y^{-1-\theta}\dd y$ again. Since the intensity is a product, after ranking the first coordinates the marks are independent with law $c^{-1}\e^{\theta f}P$ and independent of the ranked points. If each mark also carries an independent subtree and $f$ depends only on the first component of the mark, the conditional law of the subtree given the first component is unchanged. \proofstep{2}{The totals.} The sum $S_\theta$ of the points of one process satisfies $\E \e^{-uS_\theta}=\exp\{-\Gamma(1-\theta)u^\theta\}$ for $u\ge0$, so it is positive and finite almost surely. The identities \begin{align*} x^{-a}&=\frac1{\Gamma(a)}\int_0^\infty u^{a-1}\e^{-ux}\dd u\quad(a>0), \\ x^a&=\frac a{\Gamma(1-a)}\int_0^\infty(1-\e^{-ux})u^{-a-1}\dd u \quad(00$, let the mark of each node $\nu$ of depth $l$ be a standard Gaussian variable $G_\nu$ (the other marks play no role), and take $X_k(e_{[\beta]})=sG_{\beta|l}$. Then $X_m=sG_{\beta|l}$ for $m\ge l$ and $X_{l-1}=\theta_ls^2/2$, so $\mathsf K_l$ is the law $N(\theta_ls,1)$, and \eqref{I:up-onepath} gives \[ \E\sum_{\abs\nu=l}\omega_\nu G_\nu=\theta_ls, \qquad \omega_\nu=\sum_{\beta|l=\nu}\omega_\beta . \] The sum over nodes is absolutely integrable, by \eqref{I:up-onepath} applied to the path function $\abs{G_{\beta|l}}$. Conditionally on all other variables, $\omega_\nu=A\e^{sG_\nu}/(A\e^{sG_\nu}+B)$ with $A,B$ independent of $G_\nu$, so Gaussian integration by parts gives $\E[G_\nu\omega_\nu]=s\,\E[\omega_\nu(1-\omega_\nu)]$. Summing over $\nu$ and using $\sum_\nu\omega_\nu=1$ gives $\E\sum_\nu\omega_\nu^2=1-\theta_l$. By \eqref{I:up-twopath} with the coefficients $\bar w_m$ and $\varphi=1$, the left side equals $\sum_{m\ge l}\bar w_m$. Hence $\sum_{m\ge l}\bar w_m=1-\theta_l$ for $1\le l\le k$, and also for $l=0$ since $\sum_m\bar w_m=1$. Taking differences gives $\bar w_l=\theta_{l+1}-\theta_l=w_l$. \end{proof} We write $\langle\cdot\rangle$ for averages under the Gibbs weights and their independent replicas, that is, leaves $\beta^1,\beta^2,\ldots$ sampled independently from $(\omega_\beta)$. Conditional averages given $\beta^1\wedge\beta^2=l$ are taken with respect to the probability $\E\langle\,\cdot\,\1_{\{\beta^1\wedge\beta^2=l\}}\rangle/w_l$. \begin{corollary}\label{I:up-cmconsequences} Let $f$ be a bounded function of a path of marks with values in a Euclidean space, and let $F_l=\E_{\mathsf K}[f\mid e_0,\ldots,e_l]$. Then \[ \E\big\langle f(e_{[\beta^1]})\cdot f(e_{[\beta^2]})\bigm| \beta^1\wedge\beta^2=l\big\rangle =\E_{\mathsf K}\norm{F_l}^2, \] and the right side is nonnegative and nondecreasing in $l$. Moreover, \begin{equation}\label{I:up-rootvariance} \E[\log W\mid e_0]=X_0(e_0),\qquad \Var(\log W\mid e_0)\le4\Var(\log\Pi). \end{equation} \end{corollary} \begin{proof} In \eqref{I:up-twopath}, condition on the common part of the two paths. Their continuations are independent, so the conditional expectation of $f(e)\cdot f(e')$ is $\norm{F_l}^2$. The sequence $(F_l)$ is a martingale for the chain, so $\E_{\mathsf K}\norm{F_l}^2$ is nondecreasing by conditional Jensen. For \eqref{I:up-rootvariance}, take conditional expectations in \eqref{I:up-stableidentity}, using $\Pi'\overset d=\Pi$ given $e_0$. For the variance, center both logarithms at $\E\log\Pi$ and use $(x-y)^2\le2x^2+2y^2$; independence of $\Pi$ and $\Pi'$ is not needed. \end{proof} In \Cref{I:up-derivatives}\textup{(iv)} we apply the monotonicity in $l$ to the response overlap of the activation. \section{Gaussian cascades}\label{I:up:gaussian-sec} Both bounds on the free energy use one explicit computation: a Gaussian field on a cascade, integrated against a Gaussian weight at each leaf. In this chapter it bounds the spherical term (\Cref{I:up-sphere}); in \Cref{I:sec:lower} it evaluates the cavity term. The useful observation is that the field along a sampled path, divided by its precision, is a Gaussian martingale. Its variances give every overlap of the added coordinates, and the normalization of their squared norm identifies the value with a Crisanti--Sommers entropy. Let $\mathcal K=\{p\in\R^{k+1}:0\le p_0\le\cdots\le p_k\}$, and put $\Delta p_0=p_0$ and $\Delta p_l=p_l-p_{l-1}$ for $1\le l\le k$. For $x\in\mathcal K$ with $x_k<1$ let \[ \gamma_x(q)=\sum_{l=0}^kw_l\1_{\{q\ge x_l\}},\qquad \lambda_{\gamma_x}(s)=\int_s^1\gamma_x(q)\dd q=1-\sum_{l=0}^kw_l\max(x_l,s); \] $\gamma_x\in\cU$ is the order parameter with atoms of mass $w_l$ at $x_l$. Fix $n\ge1$ and $p\in\mathcal K$. Let $(y_\nu)$ be independent standard Gaussian vectors in $\R^n$ indexed by all nodes $\nu$ of the tree, including the root, independent of the cascade, and put \[ Y_\beta=\sum_{l=0}^k\sqrt{\Delta p_l}\,y_{\beta|l},\qquad Y_{\beta,l}=\sum_{j=0}^l\sqrt{\Delta p_j}\,y_{\beta|j}\qquad(0\le l\le k), \] for a leaf $\beta$. The coordinates of $Y$ are independent, and each has covariance $p_{\beta\wedge\beta'}$ between the leaves $\beta$ and $\beta'$. We call $Y$ the \emph{Gaussian cascade field with levels $p$}. Summation by parts gives \begin{equation}\label{I:gc:sbp} \sum_{l=1}^k\theta_l\Delta p_l=p_k-\sum_{l=0}^kw_lp_l . \end{equation} For $b>\sum_{l=1}^k\theta_l\Delta p_l$ define, for $0\le l\le k$, \begin{equation}\label{I:gc:parameters} d_l=b-\sum_{j=l+1}^k\theta_j\Delta p_j,\qquad m_l=\frac{p_0}{d_0^2}+\sum_{j=1}^l\frac{\Delta p_j}{d_{j-1}d_j},\qquad \cG(b,p)=\frac{p_0}{2d_0}+\frac12\sum_{j=1}^k\frac1{\theta_j}\log\frac{d_j}{d_{j-1}}, \end{equation} and put $\varrho=m_k+1/b$. Since $d_l-d_{l-1}=\theta_l\Delta p_l\ge0$ and $d_0=b-p_k+\sum_lw_lp_l>0$, we have $00$, the leaf function of \eqref{I:gc:value} is $X_k=\norm{x_k}^2/(2b)-\frac n2\log b$, and that of \eqref{I:up-quadraticvalue} is $\norm{x_k}^2/(2b)$. For one coordinate, $\theta\in(0,1)$, $\Delta\ge0$, $d>\theta\Delta$, and a standard Gaussian variable $G$, completing the square gives \[ \frac1\theta\log\E\exp\Big(\frac{\theta(x+\sqrt\Delta\,G)^2}{2d}\Big) =\frac{x^2}{2(d-\theta\Delta)}+\frac1{2\theta}\log\frac d{d-\theta\Delta}. \] Since $d_l-\theta_l\Delta p_l=d_{l-1}>0$, backward induction in \eqref{I:up-recursion} shows that $X_l(e_0,\ldots,e_l)=\norm{x_l}^2/(2d_l)+a_l$ with $a_{l-1}=a_l+\frac n{2\theta_l}\log(d_l/d_{l-1})$, where $a_k=-\frac n2\log b$ for \eqref{I:gc:value} and $a_k=0$ for \eqref{I:up-quadraticvalue}. All values are finite, so \Cref{I:up-cascade} applies. By \eqref{I:up-stableidentity}, $n$ times the left side of \eqref{I:gc:value} equals $\E X_0(e_0)=np_0/(2d_0)+a_0=n[\cG(b,p)-\frac12\log b]$, and the same computation with $a_k=0$ gives \eqref{I:up-quadraticvalue}. This proves (i). \proofstep{2}{The law of a sampled path.} Under $\cM$ the leaf $\beta^1$ is sampled from the Gibbs weights of \Cref{I:up-cascade} with the leaf function $X_k$ of \eqref{I:gc:value}, and, given the leaf and $Y$, the vector $\varepsilon^1$ has density proportional to $\e^{\varepsilon\cdot Y_{\beta^1}-b\norm\varepsilon^2/2}$ with respect to Lebesgue measure, that is, the law $N(Y_{\beta^1}/b,b^{-1}I_n)$. By \eqref{I:up-recursion}, the kernel $\mathsf K_l(\dd e_l\mid e_0,\ldots,e_{l-1})$ has density proportional to $\exp\{\theta_l\norm{x_{l-1}+e_l}^2/(2d_l)-\norm{e_l}^2/(2\Delta p_l)\}$. The coefficient of $-\norm{e_l}^2/2$ in the exponent is $1/\Delta p_l-\theta_l/d_l=d_{l-1}/(d_l\Delta p_l)$, so $\mathsf K_l$ is the Gaussian law with mean $\theta_l\Delta p_l\,x_{l-1}/d_{l-1}$ and covariance $(\Delta p_ld_l/d_{l-1})I_n$ (the point mass at $0$ if $\Delta p_l=0$). Since $d_{l-1}+\theta_l\Delta p_l=d_l$, under the chain \[ x_l=\frac{d_l}{d_{l-1}}\,x_{l-1}+\Big(\frac{\Delta p_ld_l}{d_{l-1}}\Big)^{1/2}\xi_l, \qquad\text{hence}\qquad \frac{x_l}{d_l}=\frac{x_{l-1}}{d_{l-1}}+\Big(\frac{\Delta p_l}{d_{l-1}d_l}\Big)^{1/2}\xi_l, \] with independent standard Gaussian vectors $\xi_l$ that are independent of the root mark. The root mark keeps its law $N(0,p_0I_n)$, so $x_0/d_0\sim N(0,(p_0/d_0^2)I_n)$. By \eqref{I:up-onepath}, averaged over the root mark, the process $M_l=x_l/d_l$ along the path of $\beta^1$ has this law, which is \eqref{I:gc:martingale} by the definition of $m_l$. Since $Y_{\beta^1}/b=M_k$, the vector $\varepsilon^1$ is $M_k+b^{-1/2}\xi'$ with an independent standard Gaussian vector $\xi'$, which gives \eqref{I:gc:law}. By \eqref{I:up-twopath}, $J=l$ with probability $w_l$, and the two paths then share $x_0,\ldots,x_l$ and continue independently with the same kernels. Given $x_l$, the vectors $M_k^1$ and $M_k^2$ of the two paths are therefore independent, each with conditional mean $M_l=x_l/d_l$, and $\E_\cM[\varepsilon^1\cdot\varepsilon^2\mid J=l]=\E\norm{M_l}^2=nm_l$. All functions involved are polynomials in Gaussian vectors, so the integrability required in \Cref{I:up-cascade} holds. This proves (ii). \proofstep{3}{The duality identity.} Let $\lambda_\varrho(s)=\varrho-\sum_lw_l\max(m_l,s)$ for $s\in[0,m_k]$. This function is nonincreasing, with $\lambda_\varrho'(s)=-\sum_{l:m_l0$ on $[0,m_k]$. On $[0,m_0]$ we have $\lambda_\varrho=1/d_0$, and on $[m_{j-1},m_j]$ we have $(1/\lambda_\varrho)'=\theta_j/\lambda_\varrho^2$ and $(\log\lambda_\varrho)'=-\theta_j/\lambda_\varrho$. Hence the constant piece contributes $m_0d_0^2=p_0$ and $m_0d_0=p_0/d_0$, and the $j$-th affine piece contributes $(d_j-d_{j-1})/\theta_j=\Delta p_j$ and $\theta_j^{-1}\log(d_j/d_{j-1})$, to the two integrals \begin{equation}\label{I:gc:integrals} \int_0^{m_l}\frac{\dd s}{\lambda_\varrho(s)^2}=p_l\quad(0\le l\le k), \qquad \int_0^{m_k}\frac{\dd s}{\lambda_\varrho(s)}=2\cG(b,p). \end{equation} (If $m_{j-1}=m_j$, then $\Delta p_j=0$ and $d_j=d_{j-1}$, and both contributions vanish.) Insert the first identity of \eqref{I:gc:integrals} into the left side of \eqref{I:gc:duality} and use Fubini's theorem: \[ \varrho\,p_k-\sum_lw_lm_lp_l =\int_0^{m_k}\frac{\varrho-\sum_{l:m_l>s}w_lm_l}{\lambda_\varrho(s)^2}\dd s . \] The numerator equals $\lambda_\varrho(s)+s\sum_{l:m_l\le s}w_l=\lambda_\varrho(s)-s\lambda_\varrho'(s)$ for almost every $s$, so the integrand is $(s/\lambda_\varrho(s))'$, and the integral is $m_k/\lambda_\varrho(m_k)=bm_k=b\varrho-1$. This proves \eqref{I:gc:duality} and (iii). \proofstep{4}{The entropy.} Suppose first that $\varrho=1$. Then $m_k=1-1/b<1$, $\gamma_m\in\cU$, and $\lambda_\varrho=\lambda_{\gamma_m}$ on $[0,m_k]$. Step~3 gives the first two identities in \eqref{I:gc:entropy}, and \eqref{I:gc:duality} becomes \eqref{I:up-dualityidentity}. By \eqref{I:gc:integrals} and the definition of $\CS$ with the cutoff $m_k\in[Q_{\gamma_m},1)$, \[ \CS(\gamma_m)=\frac12\int_0^{m_k}\frac{\dd s}{\lambda_{\gamma_m}(s)}+\frac12\log(1-m_k) =\cG(b,p)-\frac12\log b . \] Moreover $1/d_l=\lambda_{\gamma_m}(m_l)\le\lambda_{\gamma_m}(0)\le1$, so $b=d_k\ge d_l\ge1$; since $m_l,p_l\ge0$, \eqref{I:up-dualityidentity} gives $b-1\le p_k$, and then $m_k=1-1/b\le1-(1+p_k)^{-1}$. For general $\varrho>0$, multiply the levels and the top precision by $\varrho$. The parameters $(\varrho b,\varrho p)$ satisfy the standing condition, and by \eqref{I:gc:parameters} they have precisions $\varrho d_l$, conditional levels $m_l/\varrho$, and the same value $\cG(\varrho b,\varrho p)=\cG(b,p)$, because $p_0/d_0$ and the ratios $d_j/d_{j-1}$ do not change. Their top level satisfies $m_k/\varrho+1/(\varrho b)=1$. The case $\varrho=1$, applied to them, shows that $\gamma_{m/\varrho}\in\cU$ with largest atom $m_k/\varrho=1-1/(\varrho b)$, and \[ \CS(\gamma_{m/\varrho})=\cG(\varrho b,\varrho p)-\frac12\log(\varrho b) =\cG(b,p)-\frac12\log b-\frac12\log\varrho . \] This proves (iv). \proofstep{5}{The linear field.} Apply \Cref{I:up-cascade} with the marks of Step~1 for the levels $c$ (with $n=1$) and the leaf function $X_k=x_k$. For $\theta\in(0,1)$ and $\Delta\ge0$, $\theta^{-1}\log\E \e^{\theta(x+\sqrt\Delta\,G)}=x+\theta\Delta/2$, so $X_l=x_l+\frac12\sum_{j>l}\theta_j\Delta c_j$ and $\E X_0(e_0)=\frac12\sum_{j=1}^k\theta_j\Delta c_j$. By \eqref{I:up-stableidentity} and \eqref{I:gc:sbp} for $c$, this is \eqref{I:gc:linear}. For $k=0$ both sides of \eqref{I:gc:linear} vanish. \end{proof} \section{The interpolation and its derivatives}\label{I:up:interpolation} We now define the interpolating free energy and compute its derivatives. Throughout this section the activation $U$ is in class~\ref{I:class-A}. From now on we fix an integer $K\ge2$ and use the cascade of \Cref{I:up:cascade-sec} with \[ k=K-1,\qquad \theta_l=\frac lK\quad(0\le l\le K),\qquad\text{so that } w_l=\frac1K\quad(0\le l\le k). \] We keep writing $w_l$, which makes the role of each factor visible. We also fix the levels of the row field, \[ h=(h_0,\ldots,h_k)\in\mathcal K,\qquad h_k<1, \] so that $\gamma_h$ is the order parameter with atoms of mass $w_l$ at $h_l$. The spin field has levels $p\in\mathcal K$. At every node $\beta$ of the cascade, including the root, take independent standard Gaussian vectors $y_\beta\in\R^N$ and $z_\beta\in\R^M$, independent of $\bG$, and set, for a leaf $\beta$, \[ Y_\beta=\sum_{l=0}^k\sqrt{\Delta p_l}\,y_{\beta|l},\qquad \Xi_\beta=\sum_{l=0}^k\sqrt{\Delta h_l}\,z_{\beta|l}, \] where $\Delta h_0=h_0$ and $\Delta h_l=h_l-h_{l-1}$. Thus $Y$ and $\Xi$ are the Gaussian cascade fields with levels $p$ and $h$ in dimensions $N$ and $M$. The remaining variance $1-h_k$ of the row field is carried by an auxiliary standard Gaussian vector $\xi\in\R^M$ inside each leaf, so that each coordinate of $\Xi_\beta+\sqrt{1-h_k}\,\xi$ has variance one, as does $b_a\cdot\sigma$ for fixed $\sigma\in S_N$. This equality of variances is what makes the diagonal terms cancel when the activation is differentiated in $t$. Without the cascade, interpolations of this form were used for the perceptron by Talagrand \cite[Chapter~2]{TalagrandVolI}. \paragraph{The perturbation.} To obtain approximate Ghirlanda--Guerra identities we add a small Gaussian field whose covariance is a polynomial in the spin overlap and in the branching level. Fix an enumeration $(\vartheta_n)_{n\ge1}$ of $\mathbb Q\cap[0,1]$ and an integer $j_+\ge1$, and let $\cI=\{1,\ldots,j_+\}^3$. For $j=(j_1,j_2,j_3)\in\cI$ put \[ c_j(R,q)=(\vartheta_{j_1}R+\vartheta_{j_2}q)^{j_3}, \qquad\text{so that}\qquad \abs{c_j(R,q)}\le2^{j_3}\quad(\abs R\le1,\ 0\le q\le1). \] We use centered Gaussian fields $H^j(\sigma,\beta)$, $j\in\cI$, indexed by $\sigma\in S_N$ and leaves $\beta$, independent of each other and of all previous variables, with covariance \begin{equation}\label{I:up-pertcov} \E H^j(\sigma,\beta)H^j(\sigma',\beta')=N\,c_j\big(R,J/k\big), \qquad R=\frac{\sigma\cdot\sigma'}N,\quad J=\beta\wedge\beta' . \end{equation} This perturbation, together with the prefactor $s_N=N^{-1/16}$ used below, is the one of Mourrat \cite[(4.4)--(4.5)]{Mourrat2021}. It has the form of Panchenko's perturbation for multi-species models \cite[(24)--(28)]{Panchenko2015} (see \cite{PanchenkoBook} for one species), with the branching level $J/k$ in place of the overlap of a second species. Panchenko averages over random coefficients in the perturbation; here, as in \cite{Mourrat2021}, the perturbation coefficients are chosen at a contact point (\Cref{I:up:contact-sec}). Mourrat proves that such a field exists in \cite[Lemma~A.2]{Mourrat2021}; the construction below places its Gaussian tensors on the nodes of the cascade, so that they become part of the marks. For integrability we use the following explicit construction. For $m\ge1$ let $a_{m,0}=0$ and $a_{m,l}=(l/k)^m-((l-1)/k)^m$ for $1\le l\le k$, and let $a_{0,0}=1$ and $a_{0,l}=0$ for $l\ge1$. Thus $a_{m,l}\ge0$ and $\sum_{l\le J}a_{m,l}=(J/k)^m$ for all $m\ge0$ and $0\le J\le k$. Attach to each node $\beta$ independent standard Gaussian tensors $g^{j,i}_\beta\in(\R^N)^{\otimes i}$, $0\le i\le j_3$, and set \[ H^j(\sigma,\beta)=\sum_{i=0}^{j_3}\sum_{l=0}^k c_{j,i,l}\,N^{-i/2}\big\langle g^{j,i}_{\beta|l},\sigma^{\otimes i}\big\rangle, \qquad c_{j,i,l}^2=N\binom{j_3}{i}\vartheta_{j_1}^i \vartheta_{j_2}^{j_3-i}a_{j_3-i,l}. \] Only the nodes $\beta|l$ with $l\le\beta\wedge\beta'$ are shared by two leaves, so the covariance is \[ \sum_{i=0}^{j_3}\binom{j_3}i\vartheta_{j_1}^i\vartheta_{j_2}^{j_3-i} N^{1-i}(\sigma\cdot\sigma')^i\sum_{l\le J}a_{j_3-i,l} =N\sum_{i=0}^{j_3}\binom{j_3}i(\vartheta_{j_1}R)^i(\vartheta_{j_2}J/k)^{j_3-i}, \] which is \eqref{I:up-pertcov} by the binomial theorem. Since $a_{m,0}=0$ unless $m=0$, only the tensor of order $i=j_3$ has a root component, with $c_{j,j_3,0}^2=N\vartheta_{j_1}^{j_3}\le N$. \paragraph{The interpolating free energy.} Let $s_N=N^{-1/16}$ and $x=(x_j)_{j\in\cI}\in\R^\cI$. A configuration is a triple $(\sigma,\xi,\beta)$ with $\sigma\in S_N$, $\xi\in\R^M$, and $\beta$ a leaf, with prior $\mu_N(\dd\sigma)\,N(0,I_M)(\dd\xi)\,v_\beta$. For $t\in[0,1]$, $p\in\mathcal K$, and $x\in\R^\cI$ the Hamiltonian is \begin{align*} \mathscr H(\sigma,\xi,\beta)&=\sum_{a=1}^M U(S_a)+Y_\beta\cdot\sigma +s_N\sum_{j\in\cI}x_jH^j(\sigma,\beta),\\ S_a&=\sqrt t\,b_a\cdot\sigma+\sqrt{1-t}\,\big(\Xi_{\beta,a}+\sqrt{1-h_k}\,\xi_a\big) . \end{align*} Let $W=\sum_\beta v_\beta\iint \e^{\mathscr H(\sigma,\xi,\beta)}\,\mu_N(\dd\sigma)\,N(0,I_M)(\dd\xi)$ be the partition function, and put \[ u_N(t,p,x)=\frac1N\E\log W-\frac{p_k}2, \qquad \psi_{\sigma,N}(p)=\frac1N\E\log\sum_\beta v_\beta\int \e^{Y_\beta\cdot\sigma}\mu_N(\dd\sigma) -\frac{p_k}2 . \] The function $\psi_{\sigma,N}$ is the spherical part of $u_N$ at $t=0$ (see \eqref{I:up-initial}), and the subtraction of $p_k/2$ normalizes it: $\psi_{\sigma,N}(0)=0$. This is a marked cascade in the sense of \Cref{I:up:cascade-sec}. The root mark consists of $\bG$, $y_\emptyset$, $z_\emptyset$, and the root tensors; the mark of a node $\beta$ of depth $l\ge1$ consists of $y_\beta$, $z_\beta$, and the tensors $g^{j,i}_\beta$; and $X_k=\log\iint \e^{\mathscr H}\dd\mu_N\dd N(0,I_M)$ is the logarithm of the \emph{leaf integral}. From now on $\langle\cdot\rangle$ denotes the Gibbs measure on configurations, proportional to $v_\beta \e^{\mathscr H}$ times the product prior, together with its independent replicas $(\sigma^l,\xi^l,\beta^l)$. We write \[ R_{12}=\frac{\sigma^1\cdot\sigma^2}N,\qquad J=J_{12}=\beta^1\wedge\beta^2,\qquad X_{12}=\frac1M\sum_{a=1}^MU'(S_a^1)U'(S_a^2), \] where $S^l_a$ is $S_a$ evaluated at the $l$-th replica, and \[ r_l=\E\langle R_{12}\mid J=l\rangle,\qquad \zeta_l=\alpha_N\E\langle X_{12}\mid J=l\rangle . \] By \Cref{I:up-cascade}, $\E\langle\1_{\{J=l\}}\rangle=w_l$ whatever the parameters, and conditional averages given $J=l$ are as in \Cref{I:up:cascade-sec}. For random variables $A,B$ of two replicas we write $\Cov(A,B\mid J=l)=\E\langle AB\mid J=l\rangle-\E\langle A\mid J=l\rangle\E\langle B\mid J=l\rangle$. The next lemma collects what the comparison uses: continuity, Guerra's formula for the derivative in $t$, the right derivatives in $p$ in the directions $\1_{\ge m}$, and the monotonicity of $\zeta_l$. For $0\le m\le k$ let $\1_{\ge m}\in\mathcal K$ be the vector with coordinates $\1_{\{l\ge m\}}$; moving $p$ in this direction increases $\Delta p_m$ and leaves the other increments unchanged. \begin{lemma}\label{I:up-derivatives} The following hold. \begin{enumerate}[label=\textup{(\roman*)}] \item The function $u_N$ is continuous on $[0,1]\times\mathcal K\times\R^\cI$, and so is $\E\langle f(R_{12},J_{12})\rangle$ for every bounded measurable function $f$ on $[-1,1]\times\{0,\ldots,k\}$. \item For $00$. Locally uniformly in the parameters, $\abs{\partial\mathscr H}\le C(1+\abs\xi+\sum_m\Theta_m)$. Hence $\partial X_k=\langle\partial\mathscr H\rangle_\beta$, and, bounding $\langle\abs\xi\rangle_\beta\le \e^{\sup\mathscr H-X_k}\int\abs\xi\dd N(0,I_M)$ with \eqref{I:up-domination}, we get $\abs{\partial X_k}\le\exp\{C(1+\sum_m\Theta_m)\}$. Differentiating \eqref{I:up-recursion} gives $\partial X_{l-1}=\int\partial X_l\dd\mathsf K_l$, and the same kind of bound propagates to every level, which justifies each differentiation under the integral. Hence $\partial X_0=\E_{\mathsf K}\partial X_k$, and by \eqref{I:up-stableidentity} and \eqref{I:up-onepath} \[ \partial\E\log W=\E\,\E_{\mathsf K}\partial X_k =\E\langle\partial\mathscr H\rangle . \] \proofstep{4}{Gaussian integration by parts.} Each Gaussian coordinate at a node of depth $m$ enters the leaf integrals of all leaves below that node, and only those. Fix one such coordinate and let it vary in a compact set. Since $\abs{U'}\le L$ and $\abs\sigma=\sqrt N$, the derivative of each descendant Hamiltonian in this coordinate is bounded, uniformly over the spin and auxiliary variables. Thus the perturbed descendant leaf integrals are bounded above and below by common finite multiples of their original values, their first derivatives satisfy the same domination, and so do their second derivatives, by the bound on $U''$. The series over leaves defining the Gibbs averages therefore converge locally uniformly together with their derivatives, and by \eqref{I:up-onepath} and \eqref{I:up-domination} the sum over nodes of the expected absolute contributions is finite. This justifies Gaussian integration by parts node by node. Summing over nodes and coordinates gives, for every row $a$ and every $0\le m\le k$, \begin{align*} \E\langle y_{\beta|m}\cdot\sigma\rangle &=\sqrt{\Delta p_m}\,N\big(1-\E\langle R_{12}\1_{\{J\ge m\}}\rangle\big),\\ \E\langle z_{\beta|m,a}U'(S_a)\rangle &=\sqrt{(1-t)\Delta h_m}\big(\E\langle U''(S_a)+U'(S_a)^2\rangle -\E\langle U'(S_a^1)U'(S_a^2)\1_{\{J\ge m\}}\rangle\big),\\ \E\langle\xi_aU'(S_a)\rangle &=\sqrt{(1-t)(1-h_k)}\,\E\langle U''(S_a)+U'(S_a)^2\rangle,\\ \sum_{i=1}^N\E\langle g_{ai}U'(S_a)\sigma_i\rangle &=\sqrt{tN}\big(\E\langle U''(S_a)+U'(S_a)^2\rangle -\E\langle U'(S_a^1)U'(S_a^2)R_{12}\rangle\big). \end{align*} Here $\beta$ is the leaf of the sampled configuration. In the first line, the derivative of $\langle\sigma_i\1_{\{\beta|m=\nu\}}\rangle$ in the coordinate $y_{\nu,i}$ produces the one-replica term $\sqrt{\Delta p_m}\langle\sigma_i^2\1_{\{\beta|m=\nu\}}\rangle$ and the two-replica term $-\sqrt{\Delta p_m}\langle\sigma^1_i\sigma^2_i\1_{\{\beta^1|m=\beta^2|m=\nu\}}\rangle$; summing over $i$ and over the nodes $\nu$ of depth $m$ gives the formula, because $\abs{\sigma}^2=N$ and $\1_{\{\beta^1|m=\beta^2|m\}}=\1_{\{J\ge m\}}$. The second and fourth lines are obtained in the same way; there the factor $U''+(U')^2$ comes from differentiating $U'(S_a)\e^{U(S_a)}$. The variable $\xi$ is integrated inside the leaf measure, so in the third line only the one-replica term appears. \proofstep{5}{The derivative in $t$.} For $00$, then $\partial_{\Delta p_m}\mathscr H=y_{\beta|m}\cdot\sigma/(2\sqrt{\Delta p_m})$, and the first identity of Step~4 and Step~3 show that $\frac1N\E\log W$ has derivative $\frac12(1-\E\langle R_{12}\1_{\{J\ge m\}}\rangle)$; subtracting $\frac12$ for the term $-p_k/2$ gives \eqref{I:up-dp}. If $\Delta p_m=0$, this formula holds at $p+\epsilon\1_{\ge m}$ for every $\epsilon>0$, and by (i), applied with $f(R,l)=R\1_{\{l\ge m\}}$, both $u_N$ and the right side of \eqref{I:up-dp} are continuous as $\epsilon\downarrow0$. The mean value theorem then gives the right derivative at $\epsilon=0$. Finally, $\E\langle R_{12}\1_{\{J\ge m\}}\rangle=\sum_{l\ge m}w_lr_l$ by the definition of $r_l$. This proves (iii). For (iv), fix the root mark and apply \Cref{I:up-cmconsequences} to the path function with coordinates $\langle U'(S_a)\rangle_\beta/\sqrt M$, $a\le M$, whose norm is at most $L$. Since replicas are independent given their leaves, $\langle U'(S^1_a)U'(S^2_a)\rangle$ given the two leaves equals $\langle U'(S_a)\rangle_{\beta^1}\langle U'(S_a)\rangle_{\beta^2}$, so for fixed root mark $\E\langle X_{12}\mid J=l\rangle$ is the quantity $\E_{\mathsf K}\norm{F_l}^2$ of that corollary. It is nonnegative, nondecreasing in $l$, and at most $L^2$. These properties survive averaging over the root mark, because $w_l$ does not depend on it. This proves \eqref{I:up-monotonicity}. \end{proof} \paragraph{The endpoints.} At $t=1$, $p=0$, and $x=0$ the Hamiltonian is $\sum_aU(b_a\cdot\sigma)$, which depends neither on $\xi$ nor on the leaf, so $W=Z_N(U)$. At $t=0$ and $x=0$ the leaf integral is the product of $\int \e^{Y_\beta\cdot\sigma}\mu_N(\dd\sigma)$, which depends only on the vectors $y$, and of \[ \prod_{a=1}^M\int\exp\big\{U\big(\Xi_{\beta,a}+\sqrt{1-h_k}\,\xi_a\big)\big\}\,N(0,1)(\dd\xi_a), \] which depends only on the vectors $z$; these are independent components of the marks. For independent $F,F'$ we have $\theta^{-1}\log\E \e^{\theta(F+F')}=\theta^{-1}\log\E \e^{\theta F} +\theta^{-1}\log\E \e^{\theta F'}$, so the recursion \eqref{I:up-recursion} factorizes into a spin part and a row part, and by \eqref{I:up-stableidentity} so does $\E\log W$. The spin part is $N\psi_{\sigma,N}(p)+Np_k/2$. The row part factorizes in the same way over the rows, whose coordinates $z_{\cdot,a}$ of the node vectors are independent. For one row it is the recursion \eqref{I:pf:recursion} for the order parameter $\gamma_h$, evaluated at $(0,0)$: the auxiliary variable gives the step $\cT_{1,1-h_k}$ over $[h_k,1]$; the level $l\ge1$ gives the step $\cT_{\theta_l,\Delta h_l}$ over $[h_{l-1},h_l)$, where $\gamma_h=\theta_l$; and the root gives the heat step $\cT_{0,h_0}$ over $[0,h_0)$, where $\gamma_h=0$. Levels with $\Delta h_l=0$ give empty intervals and the identity step $\cT_{\theta_l,0}$, so they can be omitted. Hence \begin{equation}\label{I:up-initial} u_N(0,p,0)=\psi_{\sigma,N}(p)+\alpha_N\,u_{\gamma_h}(0,0;U),\qquad u_N(1,0,0)=F_N(U). \end{equation} The identification of the row part with the recursion \eqref{I:pf:recursion} is the representation of the Parisi functional through Ruelle probability cascades (see \cite[Chapter~2]{PanchenkoBook} and \cite{Arguin2007}). \section{Concentration and identities at a contact}\label{I:up:contact-sec} The comparison argument evaluates the interpolation only at maximum points of a deterministic function of the parameters. At such points we need the Ghirlanda--Guerra identities for the perturbed Gibbs measure to hold approximately. They follow from concentration of the derivatives of $\log W$ in the perturbation parameters $x$. Since the points are deterministic, concentration at each fixed parameter suffices. This section follows Steps~1--3 of the proof of \cite[Theorem~4.1]{Mourrat2021}. There a quadratic penalty around the point $(1,\ldots,1)$ \cite[(4.18)--(4.19)]{Mourrat2021} keeps the perturbation coefficients at the contact point close to one; the bound on the Hessian in $x$ at a contact point and the concentration of the free energy \cite[Proposition~4.2]{Mourrat2021} control the fluctuations of the perturbation Hamiltonians; and Gaussian integration by parts gives approximate Ghirlanda--Guerra identities \cite[(4.28)]{Mourrat2021}. \Cref{I:up-concentration,I:up-contact} carry out these steps for the present interpolation, and the penalty appears in the proof of \Cref{I:up-comparisonprop}. \begin{lemma}\label{I:up-concentration} Fix $K$, $h$, and $\cI$, and a compact subset of $[0,1]\times\mathcal K\times\R^\cI$. There is a constant $C$ such that \[ \Var\Big(\frac1N\log W\Big)\le\frac CN \] on this set. Moreover, almost surely $x\mapsto\log W$ is convex and $C^2$ on $\R^\cI$, with \begin{align} \partial_{x_j}\log W&=s_N\langle H^j\rangle,\label{I:up-xfirst}\\ \partial_{x_j}^2\log W&=s_N^2\Var_{\langle\cdot\rangle}(H^j).\label{I:up-xsecond} \end{align} The function $u_N$ is convex and differentiable in $x$, with $N\partial_{x_j}u_N=s_N\E\langle H^j\rangle$, $\abs{\partial_{x_j}u_N}\le2^{j_++1}N^{-1/8}\abs{x_j}$, and, with $C_{j_+}=2^{j_+}$, \begin{equation}\label{I:up-smallpert} \abs{u_N(t,p,x)-u_N(t,p,0)}\le C_{j_+} N^{-1/8}\abs x^2\qquad ((t,p,x)\in[0,1]\times\mathcal K\times\R^\cI). \end{equation} \end{lemma} \begin{proof} We divide the proof into three steps. \proofstep{1}{The variance.} By the law of total variance, $\Var(\log W)=\E\Var(\log W\mid e_0)+\Var(\E[\log W\mid e_0])$. By \eqref{I:up-rootvariance} the first term is at most $4\Var(\log\Pi)$, a constant depending only on the cascade, and $\E[\log W\mid e_0]=X_0(e_0)$ is a function of the root Gaussian variables $\bG$, $y_\emptyset$, $z_\emptyset$, and the root tensors. As in Steps~2 and~3 of the proof of \Cref{I:up-derivatives}, applied to these variables (the derivatives of $\mathscr H$ in them are bounded), $X_0$ is continuously differentiable in them and its gradient is the $\E_{\mathsf K}$-average of the gradient of the leaf logarithm, so by Jensen's inequality its squared norm is at most the $\E_{\mathsf K}$-average of the squared norm of the leaf gradient. The derivative of the leaf logarithm in $g_{ai}$ is $\sqrt{t/N}\,\langle U'(S_a)\sigma_i\rangle_\beta$; by Jensen's inequality and $\abs\sigma^2=N$ the sum of its squares over $a,i$ is at most $(t/N)MNL^2=Nt\alpha_NL^2$. The derivative in $y_\emptyset$ is $\sqrt{p_0}\langle\sigma\rangle_\beta$, with squared norm at most $Np_0$. The derivative in $z_{\emptyset,a}$ is $\sqrt{(1-t)h_0}\,\langle U'(S_a)\rangle_\beta$, with squared norm summed over $a$ at most $N\alpha_Nh_0L^2$. The root tensor of $H^j$ has coefficient $c_{j,j_3,0}^2\le N$, and the derivative of the leaf logarithm in it has squared norm at most $s_N^2x_j^2c_{j,j_3,0}^2N^{-j_3}\abs\sigma^{2j_3}\le s_N^2Nx_j^2$. Altogether \[ \abs{\nabla X_0}^2\le N\big(t\alpha_NL^2+p_0+\alpha_Nh_0L^2+N^{-1/8}\abs x^2\big), \] and the Gaussian Poincar\'e inequality \cite[Theorem~3.20]{BoucheronLugosiMassart2013} gives $\Var(X_0)\le\E\abs{\nabla X_0}^2\le CN$ on the compact set. Hence $\Var(N^{-1}\log W)\le N^{-2}(4\Var\log\Pi+CN)\le C/N$. \proofstep{2}{The derivatives in $x$.} Let $\bar\Theta_\beta=\sum_{j\in\cI}\sum_{i,l}c_{j,i,l} \norm{g^{j,i}_{\beta|l}}$, so that $\sum_j\sup_\sigma\abs{H^j(\sigma,\beta)}\le\bar\Theta_\beta$, and let $A_\beta(x)$ be the leaf integral. For each $x_0\in\mathbb Q^\cI$ and each integer $n\ge1$, \[ \sum_\beta v_\beta A_\beta(x_0)\big(1+\bar\Theta_\beta+\bar\Theta_\beta^2\big) \e^{s_Nn\bar\Theta_\beta}<\infty \qquad\text{almost surely}. \] Indeed, the sum equals $W(x_0)$ times a Gibbs average whose expectation, by \eqref{I:up-onepath}, is an $\E_{\mathsf K}$-expectation of $(1+\bar\Theta+\bar\Theta^2)\e^{s_Nn\bar\Theta}$, which is finite by the kernel density bound of Step~1 in the proof of \Cref{I:up-derivatives}. On the intersection of these countably many events, since $A_\beta(x)\le A_\beta(x_0)\e^{s_N\abs{x-x_0}_1\bar\Theta_\beta}$, the series defining $W(x)$ and the series of its first two derivatives converge locally uniformly in $x$. Termwise differentiation gives \eqref{I:up-xfirst} and \eqref{I:up-xsecond}. More generally, the Hessian of $\log W$ in $x$ is $s_N^2$ times the covariance matrix of $(H^j)_{j\in\cI}$ under $\langle\cdot\rangle$, which is positive semidefinite, so $\log W$ is convex in $x$. For a convex function the difference quotients $(\log W(x)-\log W(x-\epsilon\mathbf e_j))/\epsilon$ and $(\log W(x+\epsilon\mathbf e_j)-\log W(x))/\epsilon$, where $\mathbf e_j$ is the $j$-th coordinate vector of $\R^\cI$, bound $\partial_{x_j}\log W$ from below and above, are integrable, and converge monotonically as $\epsilon\downarrow0$. Hence $\partial_{x_j}\E\log W=\E\partial_{x_j}\log W$, and $u_N$ is convex in $x$. A convex function on $\R^\cI$ whose partial derivatives exist is differentiable, so $u_N$ is differentiable in $x$. \proofstep{3}{Replica integration by parts.} For a bounded measurable function $f$ of the overlap arrays $(R_{ll'},J_{ll'})_{l,l'\le n}$ of $n$ replicas, Gaussian integration by parts in the tensors gives \begin{equation}\label{I:up-replicaibp} \E\langle fH^j(1)\rangle=s_Nx_jN \bigg(\sum_{l=1}^n\E\big\langle f\,c_j(R_{1l},J_{1l}/k)\big\rangle -n\E\big\langle f\,c_j(R_{1,n+1},J_{1,n+1}/k)\big\rangle\bigg), \end{equation} where $H^j(1)=H^j(\sigma^1,\beta^1)$. To justify the exchange of the sum over nodes with the expectation, differentiate with respect to one tensor $g^{j,i}_\nu$ at one node $\nu$ of depth $m$. Each replica $l$ contributes the indicator that $\beta^l$ descends from $\nu$, and contracting the two tensors gives $N^iR_{1l}^i$. Summing the indicators over the nodes of depth $m$ gives $\1_{\{J_{1l}\ge m\}}$, and summing the squared coefficients over $i$ and $m$ gives the covariance \eqref{I:up-pertcov}. The expected absolute sum over nodes is bounded by a constant times $\E\langle\norm{g^{j,i}_{\beta|m}}\rangle$, which is finite by \eqref{I:up-onepath} and \eqref{I:up-domination}. Taking $n=1$ and $f=1$ in \eqref{I:up-replicaibp}, and using $\abs{c_j}\le2^{j_3}$, gives $\abs{\E\langle H^j\rangle}\le2^{j_3+1}s_N\abs{x_j}N$, hence $\abs{\partial_{x_j}u_N}=s_NN^{-1}\abs{\E\langle H^j\rangle}\le2^{j_3+1}N^{-1/8}\abs{x_j}$. Integrating along the segment from $0$ to $x$ gives \eqref{I:up-smallpert}, since $j_3\le j_+$. \end{proof} For two families of overlap arrays $A^{ll'}$, $B^{ll'}$, a test function $f$ of the arrays of the first $n$ replicas, and a function $c$ of two real variables, the \emph{Ghirlanda--Guerra discrepancy} is \[ \mathfrak D_n(f,c)=\E\langle f\,c(A^{1,n+1},B^{1,n+1})\rangle -\frac1n\E\langle f\rangle\E\langle c(A^{12},B^{12})\rangle -\frac1n\sum_{l=2}^n\E\langle f\,c(A^{1l},B^{1l})\rangle . \] The Ghirlanda--Guerra identities \cite{GhirlandaGuerra1998} state that $\mathfrak D_n(f,c)=0$. For $c=c_j$ this is the form used in \cite[(4.28) and (5.6)]{Mourrat2021}, and it is the analogue, for two overlap arrays, of the multi-species discrepancy in \cite[(31)]{Panchenko2015}. \begin{lemma}\label{I:up-contact} Fix a compact set of parameters $(t,p)\in[0,1]\times\mathcal K$. There is a constant $C$ with the following property. Suppose that $(t,p)$ lies in this set, that $\abs{x-\mathbf 1}\le1/2$, where $\mathbf 1\in\R^\cI$ has all coordinates equal to one, and that for every $y\in\R^\cI$ \begin{equation}\label{I:up-quadraticcontact} 0\le u_N(x+y)-u_N(x)-\nabla_xu_N(x)\cdot y\le\abs y^2, \end{equation} where $u_N(x)$ abbreviates $u_N(t,p,x)$. Then for $A=R$ and $B=J/k$, every $j\in\cI$, every $n\ge1$, and every measurable function $f$ of the overlap arrays of $n$ replicas with $\abs f\le1$, \[ \abs{\mathfrak D_n(f,c_j)}\le CN^{-1/8}. \] \end{lemma} \begin{proof} We show that $H^j$ has small fluctuations, both under the Gibbs measure and over the disorder, at the scale relevant to \eqref{I:up-replicaibp}. We divide the proof into three steps. \proofstep{1}{Thermal fluctuation.} Adding \eqref{I:up-quadraticcontact} at $y=\pm\epsilon\mathbf e_j$ gives $u_N(x+\epsilon\mathbf e_j)+u_N(x-\epsilon\mathbf e_j)-2u_N(x)\le2\epsilon^2$, and the expectation of the corresponding second difference of $\log W$ is $N$ times the left side. This second difference is nonnegative by convexity, and, divided by $\epsilon^2$, it converges to $s_N^2\Var_{\langle\cdot\rangle}(H^j)$ by \eqref{I:up-xsecond}. Fatou's lemma gives $\E\Var_{\langle\cdot\rangle}(H^j)\le2Ns_N^{-2}=2N^{9/8}$. \proofstep{2}{Disorder fluctuation.} Put $\widehat u_N=N^{-1}\log W-p_k/2$, so that $u_N=\E\widehat u_N$, and fix $\epsilon\in(0,1]$. Let $\Delta=\max\abs{\widehat u_N(z)-u_N(z)}$ over the three points $z\in\{x,x+\epsilon\mathbf e_j,x-\epsilon\mathbf e_j\}$. By convexity of $\widehat u_N$ in $x_j$ and the upper bound in \eqref{I:up-quadraticcontact}, \[ \partial_j\widehat u_N(x)\le\frac{\widehat u_N(x+\epsilon\mathbf e_j)-\widehat u_N(x)}\epsilon \le\frac{u_N(x+\epsilon\mathbf e_j)-u_N(x)+2\Delta}\epsilon \le\partial_ju_N(x)+\epsilon+\frac{2\Delta}\epsilon, \] and in the same way, with the secant through $x-\epsilon\mathbf e_j$, $\partial_j\widehat u_N(x)\ge\partial_ju_N(x)-\epsilon-2\Delta/\epsilon$. The three points are deterministic and lie in a fixed compact set, so \Cref{I:up-concentration} gives $\E\Delta^2\le C/N$. With $\epsilon=N^{-1/4}$, \[ \E\big(\partial_j\widehat u_N(x)-\partial_ju_N(x)\big)^2 \le2\epsilon^2+\frac{8\E\Delta^2}{\epsilon^2}\le CN^{-1/2}. \] By \eqref{I:up-xfirst}, $\partial_j\widehat u_N=N^{-1}s_N\langle H^j\rangle$ and $\partial_ju_N=N^{-1}s_N\E\langle H^j\rangle$, so $\E(\langle H^j\rangle-\E\langle H^j\rangle)^2\le N^2s_N^{-2}CN^{-1/2}=CN^{13/8}$. Adding the thermal fluctuation, $\E\langle(H^j-\E\langle H^j\rangle)^2\rangle\le CN^{13/8}$. \proofstep{3}{Conclusion.} Subtract from \eqref{I:up-replicaibp} for $n$ replicas the same identity with $n=1$ and $f=1$, multiplied by $\E\langle f\rangle$. The terms with $l=1$ cancel, because $c_j(R_{11},J_{11}/k)=c_j(1,1)$ is a constant, and what remains is \[ \E\big\langle f\big(H^j(1)-\E\langle H^j\rangle\big)\big\rangle =-s_Nx_jN\,n\,\mathfrak D_n(f,c_j). \] By the Cauchy--Schwarz inequality the left side is at most $CN^{13/16}$. Dividing by $ns_Nx_jN\ge\frac12N^{15/16}$, which uses $x_j\ge1/2$, gives $\abs{\mathfrak D_n(f,c_j)}\le CN^{-1/8}$. \end{proof} \section{Synchronization}\label{I:up:sync-sec} We use Panchenko's synchronization of overlaps, in the quantitative form proved by Mourrat. Panchenko proved that, under the Ghirlanda--Guerra identities, the overlap of each species in a multi-species model is a nondecreasing Lipschitz function of the total overlap \cite[Theorem~4]{Panchenko2015}. The proof rests on his theorem that the Ghirlanda--Guerra identities imply ultrametricity \cite{Panchenko2013Ultrametricity}. He proved the analogue for vector spins in \cite{Panchenko2018Vector}. Mourrat's synchronization theorem \cite[Theorem~5.3]{Mourrat2021} is a variant for two overlap arrays under approximate identities. The following form of it bounds the conditional variance of one overlap given the other, once approximate Ghirlanda--Guerra identities hold for a sufficiently rich finite family of test functions. \Cref{I:sec:lower} uses the same synchronization in the lower bound. \begin{lemma}\label{I:up-sync} Fix an enumeration $(\vartheta_n)$ of $\mathbb Q\cap[0,1]$. For every $\varepsilon>0$ there is an integer $n_\varepsilon\ge1$ with the following property. Let a random probability measure be supported on the product of the unit balls of two Hilbert spaces, and let $A^{ll'}$ and $B^{ll'}$ be the two inner-product arrays of independent samples from it. Suppose that \[ \abs{\mathfrak D_n(f,c_j)}\le\frac1{n_\varepsilon} \qquad\text{for all }n\le n_\varepsilon\text{ and }j\in\{1,\ldots,n_\varepsilon\}^3, \] and every continuous test function $f$ of the arrays of $n$ samples with $\abs f\le1$, where $c_j(u,v)=(\vartheta_{j_1}u+\vartheta_{j_2}v)^{j_3}$. If the averaged law of $B^{12}$ is $K^{-1}\sum_{l=1}^K\delta_{q_l}$, where $-1=q_00$ in place of $1/n_\varepsilon$ and with the conditions required for $n,j_1,j_2,j_3\le\lfloor\delta^{-1}\rfloor$; its hypothesis on the test functions and on the law of $B^{12}$ is the one above, and $\E\langle A^{12}\mid B^{12}\rangle$ is the conditional expectation under the averaged measure. Our form follows by taking $n_\varepsilon=\lceil\delta^{-1}\rceil$, because then $1/n_\varepsilon\le\delta$ and $\lfloor\delta^{-1}\rfloor\le n_\varepsilon$. \end{proof} We apply \Cref{I:up-sync} with the following choices. Recall that $K=k+1$ and $w_l=1/K$ for $0\le l\le k$. The maps $\sigma\mapsto\sigma/\sqrt N$ and $\beta\mapsto k^{-1/2}\sum_{l=1}^k\mathbf u_{\beta|l}$, where $(\mathbf u_\nu)$ is an orthonormal family indexed by the nodes of depth at least one, represent $R$ and $J/k$ as inner products of unit vectors, since $k^{-1}\#\{1\le l\le k:\beta|l=\beta'|l\}=(\beta\wedge\beta')/k$. Up to the factor $k^{-1/2}$, this is the embedding of \cite[(A.7)]{Mourrat2021}. The image of the Gibbs measure under the product of these maps is a random probability measure on the product of two unit balls. By \Cref{I:up-cascade}, $J/k$ has averaged law exactly uniform on $\{0,1/k,\ldots,1\}$, for every value of the parameters, including the perturbation. In the notation of \Cref{I:up-sync} this is $q_l=(l-1)/k$ for $1\le l\le K$, so the gaps $q_{l+1}-q_l$ are $1$ (for $l=0$) and $1/k$, and the right side of \eqref{I:up-syncbound} is at most $12/K+\varepsilon K^3$. We choose the parameters in the following order. Given $K$, let $\varepsilon=K^{-4}$, let $n_K=n_\varepsilon$ be given by \Cref{I:up-sync}, and take $j_+=n_K$, so that the functions $c_j$, $j\in\cI$, are exactly those of \Cref{I:up-sync}. At any point satisfying the hypotheses of \Cref{I:up-contact}, and for $N$ so large that $CN^{-1/8}\le1/n_K$, the hypotheses of \Cref{I:up-sync} hold. The left side of \eqref{I:up-syncbound} equals $\sum_lw_l\Var(R_{12}\mid J=l)$, since $\E\langle R_{12}\mid J\rangle=r_J$, and $12/K+K^{-4}K^3=13/K$. Hence, with \begin{equation}\label{I:up-defect} \cE_N=\frac12\sum_{l=0}^kw_l\Cov\big(R_{12},\alpha_NX_{12}\bigm|J=l\big) \qquad\text{and}\qquad \eta_N=\frac{\alpha_NL^2}2\sqrt{\frac{13}K}, \end{equation} we have $\sum_lw_l\Var(R_{12}\mid J=l)\le13/K$ and $\abs{\cE_N}\le\eta_N$ at such a point. The second bound follows from the first: since $\abs{X_{12}}\le L^2$, the conditional Cauchy--Schwarz inequality gives $\abs{\Cov(R_{12},\alpha_NX_{12}\mid J=l)}\le\alpha_NL^2\Var(R_{12}\mid J=l)^{1/2}$, and the Cauchy--Schwarz inequality over $l$, with $\sum_lw_l=1$, gives $\sum_lw_l\Var(R_{12}\mid J=l)^{1/2}\le(13/K)^{1/2}$. By \eqref{I:up-dt}, \begin{equation}\label{I:up-dtdefect} \partial_tu_N=-\frac12\sum_{l=0}^kw_l(r_l-h_l)\zeta_l-\cE_N . \end{equation} The tolerance $\varepsilon=K^{-4}$ and the bound $13/K$ are those of Mourrat \cite[(4.31)]{Mourrat2021}. There the conditional variances of both overlaps enter. Here only the spin overlap is synchronized with $J$, and the response overlap enters through the bound $\abs{X_{12}}\le L^2$, which gives the square root in $\eta_N$. \section{The spherical term}\label{I:up:sphere-sec} At $t=0$ the spins interact only with the cascade field $Y$, through the term $\psi_{\sigma,N}(p)$ of \eqref{I:up-initial}. This is the only geometric term in the comparison. It enters through the maximum over $p$ of $\psi_{\sigma,N}(p)+\frac12\sum_lw_lh_lp_l$, and we show that this maximum is at most $\CS(\gamma_h)+o_N(1)$. The idea is to replace the uniform measure on the sphere by a Gaussian measure with a well-chosen variance $1/b$, for which the cascade recursion is computed exactly by \Cref{I:gc:finite}. This replacement, at the cost $(b-1-\log b)/2$, is the large deviation argument behind the Parisi formula for spherical models \cite{Talagrand2006Spherical}; see also \cite{Chen2013,Mourrat2022CJM}. Here only an upper bound is needed, and it holds for each $N$ with an error of order $\log N/N$. \begin{lemma}\label{I:up-sphere} The following hold. \begin{enumerate}[label=\textup{(\roman*)}] \item For every $N$ and every $p\in\mathcal K$, $\psi_{\sigma,N}(p)\le\sqrt{p_k}-\frac12\sum_lw_lp_l$. \item For $h\in\mathcal K$ with $h_k<1$, \begin{equation}\label{I:up-envelope} \bar\psi_N:=\sup_{p\in\mathcal K} \Big\{\psi_{\sigma,N}(p)+\frac12\sum_{l=0}^kw_lh_lp_l\Big\} \le\CS(\gamma_h)+o_N(1), \end{equation} where $o_N(1)$ depends only on $K$ and $h$. \end{enumerate} \end{lemma} \begin{proof} Part (ii) rests on part (i). \proofpart{Proof of (i).} The leaf logarithm $\log\int \e^{Y_\beta\cdot\sigma}\dd\mu_N$ is $\sqrt{N\Delta p_l}$-Lipschitz in the Gaussian vector $y_{\beta|l}$, since its gradient is $\sqrt{\Delta p_l}\langle\sigma\rangle_\beta$. Differentiating \eqref{I:up-recursion} shows that each $X_{l-1}$ keeps these Lipschitz bounds in the earlier vectors, since its gradient is a $\mathsf K_l$-average. Gaussian concentration in the form $\E \e^{\theta(F-\E F)}\le \e^{\theta^2\ell^2/2}$ for an $\ell$-Lipschitz function $F$ of a standard Gaussian vector \cite[Theorem~5.5]{BoucheronLugosiMassart2013}, applied at each level $l=k,\ldots,1$, shows that $X_{l-1}$ exceeds the plain average of $X_l$ over $y_{\beta|l}$ by at most $N\theta_l\Delta p_l/2$. The root is averaged without a logarithm. Therefore, by \eqref{I:up-stableidentity}, \[ \frac1N\E\log\sum_\beta v_\beta\int \e^{Y_\beta\cdot\sigma}\dd\mu_N \le\frac1N\E\log\int \e^{Y\cdot\sigma}\dd\mu_N+\frac12\sum_{l=1}^k\theta_l\Delta p_l \le\sqrt{p_k}+\frac12\sum_{l=1}^k\theta_l\Delta p_l , \] where $Y\sim N(0,p_kI_N)$, and we used $Y\cdot\sigma\le\sqrt N\abs Y$ and $\E\abs Y\le\sqrt{Np_k}$. By \eqref{I:gc:sbp} this proves (i). \proofpart{Proof of (ii).} We divide the proof into three steps. \proofstep{1}{Localization.} By (i), since $h_l\le h_k$ and $p_l\ge0$, \[ \psi_{\sigma,N}(p)+\tfrac12\sum_lw_lh_lp_l \le\sqrt{p_k}-\tfrac12\sum_lw_l(1-h_l)p_l \le\sqrt{p_k}-\tfrac12w_k(1-h_k)p_k . \] The value at $p=0$ is zero, and the right side is negative when $p_k>\bar p:=4/[w_k(1-h_k)]^2$. So the supremum may be restricted to the set $\{p\in\mathcal K:p_k\le\bar p\}$, and we prove the bound uniformly over this set. Fix such a $p$. \proofstep{2}{Choice of the variance.} In the notation of \eqref{I:gc:parameters} with $n=N$, as $b$ increases every $d_l$ increases, so $m_k+1/b$ decreases continuously. It tends to zero as $b\to\infty$, and to infinity as $b\downarrow\sum_{j\ge1}\theta_j\Delta p_j$: if $p_0>0$, the term $p_0/d_0^2$ diverges; if $p_0=0$ and $p\ne0$, then for the first index $j_*$ with $\Delta p_{j_*}>0$ we have $d_{j_*-1}=b-\sum_{j\ge1}\theta_j\Delta p_j$, and the $j_*$-th term of $m_k$ diverges; if $p=0$, the term $1/b$ diverges. We choose $b$ as the unique solution of $m_k+1/b=1$, so that \Cref{I:gc:finite}\textup{(iv)} with $\varrho=1$ applies. In particular $b\ge1$. Also $b<2(\bar p+1)$: since $d_l\ge d_0\ge b-p_k\ge b-\bar p$, we have $m_k\le p_k/(b-\bar p)^2\le\bar p/(b-\bar p)^2$ for $b>\bar p$, and at $b=2(\bar p+1)$ this plus $1/b$ is at most $\frac18+\frac12<1$, because $(\bar p+2)^2\ge8\bar p$. \proofstep{3}{Comparison with the sphere.} For fixed $Y$, the average of $\e^{\rho Y\cdot\sigma}$ over $\sigma\in S_N$ is nondecreasing in $\rho\ge0$: by the symmetry $\sigma\mapsto-\sigma$ it equals the average of $\cosh(\rho Y\cdot\sigma)$, whose derivative in $\rho$ is nonnegative. If $\varepsilon\sim N(0,b^{-1}I_N)$, then $\varepsilon=\abs\varepsilon N^{-1/2}\sigma$ with $\sigma$ uniform on $S_N$ and independent of $\abs\varepsilon$, and $b\abs\varepsilon^2$ has the chi-square law $\chi^2_N$ with $N$ degrees of freedom. Restricting $\E \e^{Y\cdot\varepsilon}=\e^{\abs Y^2/(2b)}$ to $\{\abs\varepsilon^2\ge N\}$ and using the monotonicity gives, pathwise, \[ \log\sum_\beta v_\beta\int \e^{Y_\beta\cdot\sigma}\dd\mu_N \le\log\sum_\beta v_\beta \e^{\abs{Y_\beta}^2/(2b)}-\log\Prob\{\chi_N^2\ge bN\}. \] For $N\ge2$ and $b\ge1$, \begin{equation}\label{I:up-chisquare} -\frac1N\log\Prob\{\chi_N^2\ge bN\} \le\frac{b-1-\log b}{2}+\frac{\frac12\log(\pi N)+\log b+1+\frac1{6N}}N . \end{equation} Indeed, the chi-square density $x^{N/2-1}\e^{-x/2}/(2^{N/2}\Gamma(N/2))$ is nonincreasing on $[N-2,\infty)$, so $\Prob\{\chi^2_N\ge bN\}$ is at least twice its value at $bN+2$. Then \eqref{I:up-chisquare} follows from $(bN+2)^{N/2-1}\ge(bN)^{N/2-1}$ and Stirling's bound $\Gamma(z)\le\sqrt{2\pi}z^{z-1/2}\e^{-z+1/(12z)}$ with $z=N/2$, after collecting terms. Take expectations, divide by $N$, and use \eqref{I:up-quadraticvalue} (with $n=N$), \eqref{I:gc:entropy} and \eqref{I:up-dualityidentity} in the form $\frac{b-1}2-\frac{p_k}2=-\frac12\sum_lw_lm_lp_l$. Since $1\le b<2(\bar p+1)$, this gives \[ \psi_{\sigma,N}(p)+\tfrac12\sum_lw_lh_lp_l \le\CS(\gamma_m)+\tfrac12\sum_lw_lp_l(h_l-m_l)+C_{\bar p}\frac{\log N}N . \] It remains to show that $\CS(\gamma_h)\ge\CS(\gamma_m)+\frac12\sum_lw_lp_l(h_l-m_l)$. The map $x\mapsto\CS(\gamma_x)$ is convex on the convex set $\{x\in\mathcal K:x_k<1\}$. Indeed, compute the entropies at two points $x,x'$ with a common cutoff $q'\in[\max(x_k,x'_k),1)$: for each $s0$ on $[0,q']$, and $\abs{\max(m_l+\epsilon(h_l-m_l),s)-\max(m_l,s)}\le\epsilon\abs{h_l-m_l}$. For almost every $s$, the right derivative of $\max(m_l+\epsilon(h_l-m_l),s)$ at $\epsilon=0$ is $(h_l-m_l)\1_{\{m_l>s\}}$. Dominated convergence and \eqref{I:gc:entropy} give \[ f'(0+)=\sum_l\frac{w_l}2(h_l-m_l)\int_0^{m_l}\frac{\dd s}{\lambda_{\gamma_m}(s)^2} =\frac12\sum_lw_lp_l(h_l-m_l). \] This bounds the left side of the previous display by $\CS(\gamma_h)+C_{\bar p}\log N/N$, uniformly over $p\in\mathcal K$ with $p_k\le\bar p$, and proves (ii). \end{proof} \section{The comparison}\label{I:up:comparison} We now show that, up to the synchronization error, the free energy at $t=1$ is at most the maximum over $p$ of the interpolation at $t=0$. The reason is the following. At a point where $p$ maximizes $u_N(t,\cdot\,,x)+\frac12\sum_lw_lh_lp_l$, the product term in \eqref{I:up-dtdefect} has a sign, so $\partial_tu_N$ is at most the synchronization error. A maximum principle in $t$ turns this into a bound on the growth of the maximum over $p$. Maximizing also over $x$, with a quadratic penalty, selects a perturbation at which \Cref{I:up-contact} applies. This is a finite-$N$ form of Step~5 of the proof of \cite[Theorem~4.1]{Mourrat2021}, where the first-order conditions \cite[(4.39)--(4.40)]{Mourrat2021} at a contact point and the monotonicity of \cite[Lemma~2.4]{Mourrat2021} give the sign \cite[(4.32)]{Mourrat2021} needed for the supersolution property. Here the maximum principle in $t$ for the maximum over $p$ takes the place of viscosity solutions and of the comparison principle \cite[Proposition~3.2]{Mourrat2021}. The use of first-order conditions in the levels to control the derivative in $t$ also parallels the stationarization of Stojnic's fully lifted interpolation for bilinearly indexed random processes \cite{Stojnic2023FLbilinear,Stojnic2023FLstationarization}. There the levels on both sides are chosen at stationary points along the interpolation path, and the derivative in $t$ vanishes when the overlaps concentrate. \begin{proposition}\label{I:up-comparisonprop} Let $U$ belong to class~\ref{I:class-A}, let $K\ge2$ and $h\in\mathcal K$ with $h_k<1$, let $n_K$ and $j_+=n_K$ be as in \Cref{I:up:sync-sec}, and let $\eta_N$ and $\bar\psi_N$ be given by \eqref{I:up-defect} and \eqref{I:up-envelope}. There is $N_0$, depending only on $U$, $K$, $h$, and $\alpha'$, such that for $N\ge N_0$ \begin{equation}\label{I:up-comparison} F_N(U)\le\bar\psi_N+\alpha_Nu_{\gamma_h}(0,0;U)+\eta_N+3C_{j_+}\abs\cI N^{-1/8}. \end{equation} Consequently, \begin{equation}\label{I:up-stepbound} \limsup_{N\to\infty}F_N(U)\le\cP_U(\gamma_h) +\frac{\alpha L^2}2\sqrt{\frac{13}K}. \end{equation} \end{proposition} \begin{proof} Fix $t_0\in(0,1)$ and $\varsigma\in(0,1]$, and consider on $[0,t_0]\times\mathcal K\times\R^\cI$ the function \[ \Psi(t,p,x)=u_N(t,p,x)+\frac12\sum_lw_lh_lp_l-\abs{x-\mathbf 1}^2-(\eta_N+\varsigma)t . \] We divide the proof into four steps. \proofstep{1}{Localization.} We show that $\Psi$ attains its maximum and that all maximum points lie in a compact set that does not depend on $N$, $t_0$, or $\varsigma$. We take $N\ge N_0$, with $N_0$ fixed at the end of this step; in particular \begin{enumerate}[label=\textup{(\alph*)}] \item $2C_{j_+} N^{-1/8}\le\min\{\frac12,\abs\cI^{-1}\}$, \end{enumerate} where $C_{j_+}$ is the constant of \eqref{I:up-smallpert}. Then $C_{j_+} N^{-1/8}\abs x^2\le2C_{j_+} N^{-1/8}(\abs{x-\mathbf 1}^2+\abs\cI) \le\frac12\abs{x-\mathbf 1}^2+1$. Since $U\le U_{\max}$, the leaf integral is at most $\e^{MU_{\max}}\int \e^{Y_\beta\cdot\sigma}\dd\mu_N$ when $x=0$, so $u_N(t,p,0)\le\alpha_NU_{\max}+\psi_{\sigma,N}(p)$. With \eqref{I:up-smallpert}, \Cref{I:up-sphere}\textup{(i)} and $h_l\le h_k$, \[ \Psi(t,p,x)\le\alpha'U_{\max}^++1+\sqrt{p_k}-\tfrac12w_k(1-h_k)p_k -\tfrac12\abs{x-\mathbf 1}^2 . \] On the other hand, by \eqref{I:up-smallpert}, \eqref{I:up-initial}, $\psi_{\sigma,N}(0)=0$, and the bound $u_{\gamma_h}(0,0;U)\ge\E U(G)\ge U(0)-L$ from \eqref{I:pf:quadlower}, \[ \Psi(0,0,\mathbf 1)\ge u_N(0,0,0)-C_{j_+} N^{-1/8}\abs\cI\ge-\alpha'(\abs{U(0)}+L)-1 . \] Since $w_k(1-h_k)>0$ and $0\le p_l\le p_k$ on $\mathcal K$, we have $\Psi<\Psi(0,0,\mathbf 1)$ outside $[0,t_0]\times\mathcal K_0$, for a compact set $\mathcal K_0\subset\mathcal K\times\R^\cI$ that depends only on $U$, $K$, $h$, $\cI$, and $\alpha'$. As $\Psi$ is continuous by \Cref{I:up-derivatives}\textup{(i)}, it attains its maximum, and every maximum point lies in $[0,t_0]\times\mathcal K_0$. Let $N_0$ be the least integer such that, for all $N\ge N_0$, condition (a) holds and also \begin{enumerate}[label=\textup{(\alph*)},start=2] \item $2^{j_+}N^{-1/8}\abs x\le\frac12$ for all $(p,x)\in\mathcal K_0$, and \item $CN^{-1/8}\le1/n_K$, where $C$ is the constant of \Cref{I:up-contact} for the compact set $[0,1]\times\{p:(p,x)\in\mathcal K_0\text{ for some }x\}$. \end{enumerate} Since $\mathcal K_0$, $\cI$, and $n_K$ are determined by $U$, $K$, $h$, and $\alpha'$, so is $N_0$; it does not depend on $t_0$ or $\varsigma$. \proofstep{2}{Conditions at a maximum point.} Let $(t_*,p_*,x_*)$ be a maximum point. First, $x_*$ maximizes $x\mapsto u_N(t_*,p_*,x)-\abs{x-\mathbf 1}^2$, so $\nabla_xu_N=2(x_*-\mathbf 1)$ there. The derivative bound of \Cref{I:up-concentration} gives $\abs{x_*-\mathbf 1}\le2^{j_+}N^{-1/8}\abs{x_*}$, so $\abs{x_*-\mathbf 1}\le\frac12$ by (b). Maximality in $x$ gives $u_N(x_*+y)-u_N(x_*)\le\abs{x_*+y-\mathbf 1}^2-\abs{x_*-\mathbf 1}^2 =\nabla_xu_N(x_*)\cdot y+\abs y^2$ for all $y\in\R^\cI$, and convexity of $u_N$ in $x$ gives the lower bound in \eqref{I:up-quadraticcontact}. Thus \Cref{I:up-contact} applies, and by (c) and \Cref{I:up:sync-sec}, $\abs{\cE_N}\le\eta_N$ at $(t_*,p_*,x_*)$. Second, $p_*$ maximizes $p\mapsto u_N(t_*,p,x_*)+\frac12\sum_lw_lh_lp_l$ over $\mathcal K$. Since $p_*+\epsilon\1_{\ge m}\in\mathcal K$ for $\epsilon\ge0$, the right derivative in this direction is nonpositive, and by \eqref{I:up-dp} \begin{equation}\label{I:up-foc} A_m:=\frac12\sum_{l=m}^kw_l(h_l-r_l)\le0\qquad(0\le m\le k). \end{equation} Third, $t_*$ maximizes $t\mapsto\Psi(t,p_*,x_*)$ over $[0,t_0]$. \proofstep{3}{Every maximum point has $t_*=0$.} Suppose that $t_*>0$. Since $t_*\le t_0<1$, $u_N$ is differentiable in $t$ at $(t_*,p_*,x_*)$ by \Cref{I:up-derivatives}\textup{(ii)}, and maximality in $t$ gives $\partial_tu_N(t_*,p_*,x_*)\ge\eta_N+\varsigma$. We now show the opposite inequality. Put $\zeta_{-1}=0$, so that $\zeta_l=\sum_{m\le l}(\zeta_m-\zeta_{m-1})$. Summation by parts, \eqref{I:up-monotonicity} and \eqref{I:up-foc} give \[ \sum_lw_l(r_l-h_l)\zeta_l =\sum_{m=0}^k(\zeta_m-\zeta_{m-1})\sum_{l=m}^kw_l(r_l-h_l) =-2\sum_{m=0}^k(\zeta_m-\zeta_{m-1})A_m\ge0 . \] By \eqref{I:up-dtdefect} and Step~2, $\partial_tu_N=-\frac12\sum_lw_l(r_l-h_l)\zeta_l-\cE_N\le\abs{\cE_N}\le\eta_N$, a contradiction. Hence $t_*=0$. \proofstep{4}{Conclusion.} Compare $\Psi$ at $(t_0,0,\mathbf 1)$ with its value at a maximum point, which has $t_*=0$. By \eqref{I:up-smallpert} and \eqref{I:up-initial}, \[ \begin{aligned} u_N(t_0,0,\mathbf 1)-(\eta_N+\varsigma)t_0&\le\Psi(0,p_*,x_*)\\ &\le\psi_{\sigma,N}(p_*)+\tfrac12\sum_lw_lh_lp_{*,l}+\alpha_Nu_{\gamma_h}(0,0;U)\\ &\qquad+C_{j_+} N^{-1/8}\abs{x_*}^2-\abs{x_*-\mathbf 1}^2 . \end{aligned} \] By (a), the last two terms are at most $2C_{j_+} N^{-1/8}\abs\cI$, and the first two are at most $\bar\psi_N$. Letting $t_0\uparrow1$, by continuity of $u_N$, and then $\varsigma\downarrow0$, \[ u_N(1,0,\mathbf 1)\le\bar\psi_N+\alpha_Nu_{\gamma_h}(0,0;U)+\eta_N+2C_{j_+} N^{-1/8}\abs\cI . \] Finally, $F_N(U)=u_N(1,0,0)\le u_N(1,0,\mathbf 1)+C_{j_+} N^{-1/8}\abs\cI$ by \eqref{I:up-initial} and \eqref{I:up-smallpert}. This proves \eqref{I:up-comparison}. By \Cref{I:up-sphere}\textup{(ii)}, $\bar\psi_N\le\CS(\gamma_h)+o_N(1)$, and $\alpha_N\to\alpha$, so \eqref{I:up-comparison} gives \eqref{I:up-stepbound}. \end{proof} \begin{proof}[Proof of \Cref{I:up-soft}] Let $\gamma\in\cU$ and $K\ge2$. Approximate $\gamma$ by the order parameter $\gamma_K=\lfloor K\gamma\rfloor/K$, which has $K$ atoms of mass $1/K$ at the quantiles $h_l=\inf\{q:\gamma(q)\ge(l+1)/K\}$, $0\le l\le k=K-1$. Indeed, by right-continuity $h_l\le q$ if and only if $\gamma(q)\ge(l+1)/K$, so $\gamma_h(q)=K^{-1}\#\{l:h_l\le q\}=\lfloor K\gamma(q)\rfloor/K$. Thus $h\in\mathcal K$, $h_k=Q_\gamma<1$, $\gamma-1/K<\gamma_K\le\gamma$, so that $\norm{\gamma_K-\gamma}_{L^1}\le1/K$, and $\gamma_K=\gamma=1$ on $[Q_\gamma,1]$. By \eqref{I:up-stepbound} for this $h$, and by \eqref{I:up-trialcontinuity} with $\bar q=Q_\gamma$, since $\gamma,\gamma_K\in\cU_{Q_\gamma}$, \[ \limsup_{N\to\infty}F_N(U)\le\cP_U(\gamma) +\frac1K\Big(\frac{\alpha L^2}2+\frac{Q_\gamma}{2(1-Q_\gamma)^2}\Big) +\frac{\alpha L^2}2\sqrt{\frac{13}K}. \] Letting $K\to\infty$ proves \eqref{I:up-softbound}. If $L=0$, the activation is constant and $f_N(U)=F_N(U)$, so the probability bound follows from \eqref{I:up-softbound}. Let $L>0$. The map $\bG\mapsto f_N(U)$ is Lipschitz: its derivative in $g_{ai}$ is $N^{-3/2}\langle U'(b_a\cdot\sigma)\sigma_i\rangle$, where now $\langle\cdot\rangle$ is the Gibbs measure of $Z_N(U)$, and by Jensen's inequality and $\abs\sigma^2=N$ the sum of the squares is at most $\alpha_NL^2/N$. Gaussian concentration \cite[Theorem~5.6]{BoucheronLugosiMassart2013} gives $\Prob\{f_N(U)-F_N(U)>\varepsilon/2\}\le\exp(-\varepsilon^2N/(8\alpha_NL^2))$, and $F_N(U)\le\cP_U(\gamma)+\varepsilon/2$ for large $N$ by \eqref{I:up-softbound}. The parameters and limits are chosen, with $U$ and $\gamma$ fixed, in the following order: first $K$ (and with it $h$), then the synchronization tolerance $\varepsilon=K^{-4}$, the integer $n_K$, and the perturbation family $\cI$; then, for each $N\ge N_0$, the auxiliary $t_0\uparrow1$ and $\varsigma\downarrow0$ inside the proof of \Cref{I:up-comparisonprop}; then $N\to\infty$; and finally $K\to\infty$. \end{proof} \section{Hard constraints}\label{I:up:hard-sec} We now pass to the hard wall. Recall that $V_N(\kappa)=Z_N(H_\kappa)$ with $H_\kappa=\log\1_{[\kappa,\infty)}$ and $\log0=-\infty$. \Cref{I:sec:gaps} also needs the feasible volume weighted by a bounded function of the fields, so we allow a source term from the start. \paragraph{Terminal data with a hard wall.} Let $\gamma$ be a step order parameter as in \eqref{I:pf:stepgamma}, and let $\varphi:\R\to\R$ be bounded and measurable. The terminal datum $H_\kappa+\varphi$ is bounded above by $\norm\varphi_\infty$, so the recursion \eqref{I:pf:recursion} with $T=1$ defines $u_\gamma(\cdot,\cdot\,;H_\kappa+\varphi)$, independently of the representation of $\gamma$, and we put \[ \cP_{H_\kappa+\varphi}(\gamma)=\alpha u_\gamma(0,0;H_\kappa+\varphi)+\CS(\gamma). \] The function $u_\gamma(q,x;H_\kappa)$ is finite for $q<1$: for $q\le t_n$, the last breakpoint of $\gamma$, it equals $u_\gamma(q,x;\kappa)$ by \eqref{I:pf:Hkappa}, and for $t_n\le q<1$ it equals $\log\Prob\{x+\sqrt{1-q}\,G\ge\kappa\}$ by \eqref{I:pf:flatstep}. By \eqref{I:pf:monotone}, applied in both directions, for bounded measurable $\varphi,\varphi'$ the functions $u_\gamma(\cdot,\cdot\,;H_\kappa+\varphi)$ are finite on $[0,1)\times\R$, and \begin{equation}\label{I:up-sourcecompare} \abs{u_\gamma(q,x;H_\kappa+\varphi)-u_\gamma(q,x;H_\kappa+\varphi')} \le\sup_{y\in\R}\abs{\varphi(y)-\varphi'(y)},\qquad (q,x)\in[0,1)\times\R . \end{equation} By \eqref{I:pf:Hkappa}, $\cP_{H_\kappa}(\gamma)=\cP_\kappa(\gamma)$. \paragraph{The wall approximation.} We approximate the hard wall from above by activations in class~\ref{I:class-A} that decrease to it. Let \begin{equation}\label{I:up-wall} \Omega(y)=y^+-\arctan(y^+),\qquad U_\ell(y;\kappa)=-\ell\,\Omega(\kappa-y)\qquad(\ell\ge1). \end{equation} The function $\Omega$ vanishes on $(-\infty,0]$, is positive on $(0,\infty)$, and is $C^2$ with $\Omega'(y)=(y^+)^2/(1+(y^+)^2)\in[0,1]$ and $\Omega''(y)=2y^+/(1+(y^+)^2)^2\in[0,1]$. Hence $U_\ell(\cdot\,;\kappa)$ belongs to class~\ref{I:class-A}, with $\sup U_\ell=0$ and $\abs{U_\ell'},\abs{U_\ell''}\le\ell$. Moreover, $\e^{U_\ell}=1$ on $[\kappa,\infty)$, and $\e^{U_\ell(y)}$ decreases to zero as $\ell\to\infty$ for $y<\kappa$. Thus $\e^{U_\ell}\downarrow\1_{[\kappa,\infty)}$ pointwise. \begin{corollary}\label{I:up-hardvolume} Let $\kappa\in\R$ and $\varepsilon>0$. \begin{enumerate}[label=\textup{(\roman*)}] \item For every $\gamma\in\cU$, \[ \Prob\Big\{\frac1N\log V_N(\kappa)>\cP_\kappa(\gamma)+\varepsilon\Big\}\longrightarrow0 . \] \item Let $\gamma$ be a step order parameter and let $\varphi:\R\to\R$ be bounded and Lipschitz. Then \[ \Prob\bigg\{\frac1N\log\int_{S_N(\kappa)} \exp\Big\{\sum_{a=1}^M\varphi(b_a\cdot\sigma)\Big\}\,\mu_N(\dd\sigma) >\cP_{H_\kappa+\varphi}(\gamma)+\varepsilon\bigg\}\longrightarrow0 . \] \end{enumerate} \end{corollary} Part (ii) with $\varphi=0$ is part (i) for step order parameters. \Cref{I:sec:gaps} uses part (ii) with $\varphi$ a real multiple of a bounded Lipschitz test function translated by $\kappa$, and \Cref{I:sec:hard-volume} uses part (i). \begin{proof} We prove (ii) first, since (i) is deduced from it. \proofpart{Proof of (ii).} Let $\varphi$ be $L_\varphi$-Lipschitz. The activation must be $C^2$, so we first smooth $\varphi$. Let $\chi$ be a smooth probability density supported in $[-1,1]$, and for $\tau\in(0,1]$ let $\varphi_\tau=\varphi*\chi_\tau$ with $\chi_\tau(y)=\tau^{-1}\chi(y/\tau)$. Then $\varphi_\tau$ is smooth, $\norm{\varphi_\tau}_\infty\le\norm\varphi_\infty$, and $\abs{\varphi_\tau-\varphi}\le L_\varphi\tau$. Moreover $\varphi_\tau'=\varphi'*\chi_\tau$ and $\varphi_\tau''=\varphi'*\chi_\tau'$, where $\varphi'$ is the almost everywhere derivative, so $\abs{\varphi_\tau'}\le L_\varphi$ and $\abs{\varphi_\tau''}\le L_\varphi\norm{\chi'}_{L^1}/\tau$. Hence $\widetilde U_\ell=U_\ell(\cdot\,;\kappa)+\varphi_\tau$, with $U_\ell$ from \eqref{I:up-wall}, belongs to class~\ref{I:class-A} for every $\ell$. Since $U_\ell(\cdot\,;\kappa)=0$ on $[\kappa,\infty)$ and the integrand of $Z_N(\widetilde U_\ell)$ is positive, \[ \int_{S_N(\kappa)}\e^{\sum_a\varphi(b_a\cdot\sigma)}\,\mu_N(\dd\sigma) \le \e^{ML_\varphi\tau}\int_{S_N(\kappa)}\e^{\sum_a\varphi_\tau(b_a\cdot\sigma)}\,\mu_N(\dd\sigma) \le \e^{ML_\varphi\tau}Z_N(\widetilde U_\ell). \] So the normalized logarithm in (ii) is at most $f_N(\widetilde U_\ell)+\alpha'L_\varphi\tau$. We show that $u_\gamma(0,0;\widetilde U_\ell)$ decreases to $u_\gamma(0,0;H_\kappa+\varphi_\tau)$ as $\ell\to\infty$. The terminal data satisfy $\e^{\widetilde U_\ell}\downarrow\1_{[\kappa,\infty)}\e^{\varphi_\tau} =\e^{H_\kappa+\varphi_\tau}$ pointwise, with values in $(0,\e^{\norm\varphi_\infty}]$. The recursion \eqref{I:pf:recursion} applies the same operators $\cT_{m,s}$ to both terminal data. Each is nondecreasing, so by induction over the steps the functions obtained from $\widetilde U_\ell$ decrease in $\ell$; they are bounded above by $\norm\varphi_\infty$, and their limits are the functions obtained from $H_\kappa+\varphi_\tau$, which are finite by \eqref{I:up-sourcecompare}. Indeed, at a step with $m>0$ the functions $\e^{mf_\ell}\in(0,\e^{m\norm\varphi_\infty}]$ decrease to $\e^{mf}$, and at a step with $m=0$ the functions $\norm\varphi_\infty-f_\ell\ge0$ increase to $\norm\varphi_\infty-f$; bounded and monotone convergence apply. Hence $\cP_{\widetilde U_\ell}(\gamma)\downarrow\cP_{H_\kappa+\varphi_\tau}(\gamma)$, and $\cP_{H_\kappa+\varphi_\tau}(\gamma)\le\cP_{H_\kappa+\varphi}(\gamma)+\alpha L_\varphi\tau$ by \eqref{I:up-sourcecompare}. Now choose $\tau$ with $(\alpha'+\alpha)L_\varphi\tau\le\varepsilon/3$, and then $\ell$ with $\cP_{\widetilde U_\ell}(\gamma)\le\cP_{H_\kappa+\varphi_\tau}(\gamma)+\varepsilon/3$. Then $\cP_{\widetilde U_\ell}(\gamma)+\alpha'L_\varphi\tau\le\cP_{H_\kappa+\varphi}(\gamma)+2\varepsilon/3$, and \Cref{I:up-soft}, applied to $\widetilde U_\ell$ and $\gamma$, gives \[ \Prob\bigg\{\frac1N\log\int_{S_N(\kappa)} \e^{\sum_a\varphi(b_a\cdot\sigma)}\,\mu_N(\dd\sigma) >\cP_{H_\kappa+\varphi}(\gamma)+\varepsilon\bigg\} \le\Prob\Big\{f_N(\widetilde U_\ell)>\cP_{\widetilde U_\ell}(\gamma)+\frac\varepsilon3\Big\} \longrightarrow0 . \] \proofpart{Proof of (i).} Let $\gamma\in\cU$ and $r=Q_\gamma<1$. The step order parameters $\gamma_n\in\cU_r$ of \Cref{I:pf:step} converge to $\gamma$ in $L^1([0,1])$, so $u_{\gamma_n}(0,0;\kappa)\to u_\gamma(0,0;\kappa)$ by the definition of $u_\gamma$ through \Cref{I:pf:coef-cont}, and $\CS(\gamma_n)\to\CS(\gamma)$ by \eqref{I:pf:CSlip}. Hence there is a step order parameter $\gamma'\in\cU_r$ with $\cP_\kappa(\gamma')\le\cP_\kappa(\gamma)+\varepsilon/2$. Part (ii) with $\varphi=0$, the order parameter $\gamma'$, and $\varepsilon/2$ gives the claim, since $\cP_{H_\kappa}(\gamma')=\cP_\kappa(\gamma')$. \end{proof} \section{Volume from one feasible point}\label{I:up:cap-sec} A bound on the volume does not by itself bound the largest feasible margin, because a feasible set may have zero spherical volume. \Cref{I:up-cap} shows that one feasible point at margin $\kappa$ produces exponentially small, but not superexponentially small, volume at margin $\kappa-\eta$, on a set where the fields $b_a\cdot\sigma$ change little. The set it constructs is used again in \Cref{I:sec:gaps}. The proof uses \v{S}id\'ak's inequality for a Gaussian vector whose covariance may be singular; \Cref{I:sec:hard-volume} uses the same extension. \begin{lemma}\label{I:up-sidak} Let $(\xi_1,\ldots,\xi_M)$ be a centered Gaussian vector with an arbitrary, possibly singular, covariance matrix, and let $c_1,\ldots,c_M\ge0$. Then \[ \Prob\big\{\abs{\xi_a}\le c_a\text{ for all }a\le M\big\} \ge\prod_{a=1}^M\Prob\{\abs{\xi_a}\le c_a\}. \] \end{lemma} \begin{proof} For a nonsingular covariance matrix this is \v{S}id\'ak's inequality \cite[Corollary~1]{Sidak1967}, proved independently by Khatri \cite{Khatri1967}. If $c_a=0$ and $\Var\xi_a>0$ for some $a$, the right side vanishes, and a coordinate with $c_a=0$ and $\xi_a=0$ contributes the factor one to both sides and may be removed. So we may assume that all $c_a>0$. Let $\xi'\sim N(0,I_M)$ be independent of $\xi$, and for $\epsilon>0$ put $\xi^\epsilon=\xi+\epsilon\xi'$, whose covariance matrix is nonsingular. Then $\Prob\{\abs{\xi^\epsilon_a}\le c_a\ \forall a\}\ge\prod_a\Prob\{\abs{\xi^\epsilon_a}\le c_a\}$. Let $\epsilon\downarrow0$ along a sequence. Each factor converges to $\Prob\{\abs{\xi_a}\le c_a\}$: if $\Var\xi_a>0$, because $\xi^\epsilon_a$ and $\xi_a$ are centered Gaussian with variances converging and a continuous limiting law; if $\xi_a=0$, because $\Prob\{\abs{\epsilon\xi'_a}\le c_a\}\to1$. Since $\xi^\epsilon\to\xi$ pointwise and the intervals are closed, the indicator of $\{\abs{\xi^\epsilon_a}\le c_a\ \forall a\}$ has $\limsup$ at most the indicator of $\{\abs{\xi_a}\le c_a\ \forall a\}$. Fatou's lemma, applied to one minus the indicators, proves the lemma. \end{proof} \begin{lemma}\label{I:up-cap} Fix $\alpha'<\infty$. There are constants $K_{\rm cap}\ge1$, $C_{\rm cap}<\infty$, and $N_0$, depending only on $\alpha'$, with the following property. Let $N\ge N_0$, $M\le\alpha'N$, and let $b_1,\ldots,b_M\in\R^N$ be deterministic vectors with $\abs{b_a}^2\le2$. Let $\kappa\in\R$ and $\eta\in(0,1]$, suppose that some $\sigma_0\in S_N$ satisfies $b_a\cdot\sigma_0\ge\kappa$ for all $a$, and put \[ s_0=\min\Big\{\frac\eta{2K_{\rm cap}},\sqrt{\frac\eta{2\abs\kappa+1}}\Big\}. \] There is a Borel set $T\subset\{\sigma\in S_N:b_a\cdot\sigma\ge\kappa-\eta\text{ for all }a\}$ with \begin{equation}\label{I:up:cap-set} \begin{gathered} \mu_N(T)\ge\exp\big\{-N\big(\log(2/s_0)+C_{\rm cap}\big)\big\}, \\ \abs{b_a\cdot\sigma-b_a\cdot\sigma_0}\le s_0^2\abs{b_a\cdot\sigma_0}+s_0K_{\rm cap} \quad(\sigma\in T,\ a\le M). \end{gathered} \end{equation} If $\kappa$ ranges over a compact interval $I$, then $\log(2/s_0)\le\log(1/\eta)+C_I$ with $C_I$ depending only on $\alpha'$ and $I$. \end{lemma} \begin{proof} We divide the proof into four steps. \proofstep{1}{Constants.} Let $c_0=\frac12(\log2-\frac12)>0$. For $X\sim\chi^2_d$ and $00$, then $\rho\, b_a\cdot\sigma_0\ge(1-s^2)\kappa$; if $b_a\cdot\sigma_0\ge0\ge\kappa$, the left side is nonnegative; and if $\kappa\le b_a\cdot\sigma_0<0$, then $\rho\, b_a\cdot\sigma_0\ge b_a\cdot\sigma_0$. Therefore \[ b_a\cdot\sigma\ge\kappa-s_0^2\abs\kappa-s_0K_{\rm cap}\ge\kappa-\frac\eta2-\frac\eta2, \] using $s_0^2\le\eta/(2\abs\kappa+1)$ and $s_0\le\eta/(2K_{\rm cap})$. Thus every $\sigma\in T$ is feasible at margin $\kappa-\eta$. \proofstep{4}{Volume.} On the interval of $\rho$ defining $T$ we have $1-\rho^2\ge s_0^2/4$, and the interval has length $\frac34s_0^2/(\sqrt{1-s_0^2/4}+\sqrt{1-s_0^2})\ge\frac38s_0^2$. Hence its probability is at least $\frac12(s_0/2)^{N-3}\cdot\frac38s_0^2=\frac3{2s_0}(s_0/2)^N\ge(s_0/2)^N$. For the direction, represent $\upsilon=P\xi/\abs{P\xi}$ with $\xi\sim N(0,I_N)$, so that $b_a\cdot\upsilon=(Pb_a)\cdot\xi/\abs{P\xi}$. On the event $\{\abs{P\xi}^2\ge N/4\}\cap\{\abs{(Pb_a)\cdot\xi}\le K_{\rm cap}/2\text{ for all }a\}$ we have $\sqrt N\abs{b_a\cdot\upsilon}\le K_{\rm cap}$ for all $a$. The variables $(Pb_a)\cdot\xi$ are centered and jointly Gaussian with variances $\abs{Pb_a}^2\le2$, and $\Prob\{\abs{N(0,v)}\le K_{\rm cap}/2\}$ is nonincreasing in $v$. By \Cref{I:up-sidak}, the choice of $K_{\rm cap}$, and $M\le\alpha'N$, \[ \Prob\big\{\abs{(Pb_a)\cdot\xi}\le K_{\rm cap}/2\text{ for all }a\big\} \ge\pi(K_{\rm cap})^M\ge \e^{-c_0N/4}. \] Since $\abs{P\xi}^2$ has the law $\chi^2_{N-1}$ and $N/4\le(N-1)/2$, Step~1 gives $\Prob\{\abs{P\xi}^20$. For every real $\kappa>\kc(\alpha)$, $\Prob\{\kappa_N\ge\kappa\}\to0$. \end{corollary} \begin{proof} Choose $\eta\in(0,1]$ with $\kappa-\eta>\kc$, and let $N_0$ and $C_V=\log(2/s_0)+C_{\rm cap}$ be given by \Cref{I:up-cap} for $\alpha'=\sup_N\alpha_N$, the margin $\kappa$, and this $\eta$. Since $\kappa-\eta>\kc$, the definition of $\kc$ gives $\cP_*(\kappa-\eta)=-\infty$, so there is $\gamma\in\cU$ with $\cP_{\kappa-\eta}(\gamma)<-C_V-1$. By \Cref{I:up-hardvolume}\textup{(i)} with $\varepsilon=1$, \[ \Prob\Big\{\frac1N\log V_N(\kappa-\eta)\ge-C_V\Big\} \le\Prob\Big\{\frac1N\log V_N(\kappa-\eta)>\cP_{\kappa-\eta}(\gamma)+1\Big\} \longrightarrow0 . \] On the other hand, suppose that $N\ge N_0$, $\kappa_N\ge\kappa$, and $\max_a\abs{b_a}^2\le2$. A maximizer $\sigma_0$ of $\min_ab_a\cdot\sigma$ over the compact set $S_N$ satisfies $b_a\cdot\sigma_0\ge\kappa$ for all $a$, so \Cref{I:up-cap} gives $V_N(\kappa-\eta)\ge\mu_N(T)\ge \e^{-NC_V}$. Finally, $N\abs{b_a}^2$ has the law $\chi^2_N$, and the Chernoff bound $\Prob\{\chi^2_N\ge2N\}\le \e^{-N(1-\log2)/2}$ (the upper-tail analogue of Step~1 in the proof of \Cref{I:up-cap}) gives \[ \Prob\Big\{\max_a\abs{b_a}^2>2\Big\}\le\alpha'N\e^{-N(1-\log2)/2}\longrightarrow0 . \] Hence $\Prob\{\kappa_N\ge\kappa\}\le\Prob\{N^{-1}\log V_N(\kappa-\eta)\ge-C_V\} +\Prob\{\max_a\abs{b_a}^2>2\}\to0$. \end{proof} \chapter{The lower bound}\label{I:sec:lower} In this chapter we prove the lower bound $\liminf_NF_N(U)\ge\cP_*(U)$ for smooth activations $U$ (\Cref{I:lo-lower}). Together with the upper bound of \Cref{I:sec:upper}, it gives the Parisi formula in \Cref{I:sec:parisi}. The argument is outlined in Step~2 of \Cref{I:sec:ideas}. \Cref{I:lo:coords} introduces the spherical coordinates, the Poisson row count, and the three systems that are compared, and \Cref{I:sec-overlap-input} proves the overlap representation theorem (\Cref{I:thm-external-overlap}) from the external results behind it. \Cref{I:lo:perturbations} defines the perturbation and states the perturbation, continuity, and precision lemmas, which are proved in \Cref{I:sec:lower-technical}. \Cref{I:lo:cavity-terms} reduces the increment to the row term and the coordinate term. \Cref{I:lo:cascades} evaluates them on cascades with \Cref{I:gc:finite} and removes the cutoff on $\norm\varepsilon$ there (\Cref{I:lo-positive-cont}). \Cref{I:lo:rotation-sec} proves the rotation estimate, and \Cref{I:lo:proof} proves \Cref{I:lo-lower}. The two cavity steps, adding constraints and adding coordinates, follow the cavity method for the perceptron in the replica-symmetric regime \cite{ShcherbinaTirozzi2003}, \cite[Chapters~2 and~3]{TalagrandVolI}, \cite[Chapter~8]{TalagrandVolII}. As in \cite{Chen2013,Panchenko2014Parisi}, the trial order parameter comes from the limiting overlap law of the system. Compared with W.-K.\ Chen's spherical form \cite{Chen2013} of the Aizenman--Sims--Starr scheme and with its multi-species version by Bates and Sohn \cite{BatesSohn2022EJP}, two points differ. First, in both works the cavity functional is identified with the spherical Parisi functional, as the number of cavity coordinates grows, through Talagrand's computation \cite{Talagrand2006Spherical}; here the coordinate term is an explicit Gaussian integral on cascades, and the rotation invariance of the patterns identifies it with the Crisanti--Sommers entropy. Second, the other species in the Ghirlanda--Guerra identities is not a group of coordinates, as in \cite{Panchenko2015,BatesSohn2022EJP}, but the response vector. \paragraph{Activation class.} Throughout this chapter, $U$ satisfies, for some $U_{\max}<\infty$ and $L<\infty$ (enlarged, if necessary, so that $L\ge1+\abs{U(0)}$), \begin{equation}\label{I:lo:class} U\in C^4(\R),\qquad U\le U_{\max},\qquad \abs{U'(x)}\le L(1+\abs x),\qquad \abs{U^{(j)}(x)}\le L\quad(2\le j\le4), \end{equation} and in addition either $U'$ is bounded or $U$ is concave. In the first case $U$ belongs to class~\ref{I:class-A}, and in the second to class~\ref{I:class-B}. The derivatives of orders three and four are used only in the perturbation of \Cref{I:lo:perturbations}, in the error estimates of the cavity interpolation, and in \Cref{I:lo-precision}. They control the first and second derivatives of $xU'(x)$ and $U'(x)^2$. In \Cref{I:sec:parisi} this extra smoothness is removed by mollification. By~\eqref{I:lo:class} and Taylor's formula, \begin{equation}\label{I:lo:quadratic} -C_0(1+x^2)\le U(x)\le U_{\max},\qquad C_0=2L . \end{equation} \paragraph{Conventions.} Throughout this chapter, $C$ and $c$ denote positive constants that may depend on $\alpha$, $U_{\max}$, and $L$, but not on $N$; their values may change from one occurrence to the next. A dependence on the cavity dimension $n$ or on the cutoff $\Lambda$ is indicated by a subscript, as in $C_{n,\Lambda}$ or $O_{n,\Lambda}(\cdot)$. The cavity dimension $n$ is fixed until the last step of the proof. We write $\nu_n$ for the standard Gaussian measure on $\R^n$ and $G$ for a standard Gaussian variable. The letter $a$ is a row index, and $\bG,\bG',\bG''$ denote Gaussian matrices with rows $g_a,g'_a,g''_a$. The letter $b$, with or without subscripts, denotes a precision parameter of a Gaussian integral, as in \Cref{I:gc:finite}. The rows $b_a=g_a/\sqrt N$ of \Cref{I:sec:intro} and the derivative $b=\partial_xu$ of \Cref{I:fun:finite-cutoff} are not used in this chapter. The goal of the chapter is the following theorem. \begin{theorem}\label{I:lo-lower} For every $\alpha>0$ and every activation $U$ satisfying \eqref{I:lo:class}, with either bounded $U'$ or concave $U$, \[ \liminf_{N\to\infty}F_N(U)\ge\cP_*(U) =\inf_{\gamma\in\cU}\bigl\{\alpha u_\gamma(0,0;U)+\CS(\gamma)\bigr\}. \] \end{theorem} \section{Spherical coordinates, the row count, and the three systems} \label{I:lo:coords} We first separate the added coordinates without changing the surface measure. Throughout this chapter \[ N'=N+n . \] For $\sigma\in S_N$ and $\varepsilon\in\R^n$ with $\norm\varepsilon^2\sqrt M+\sqrt N+t\bigr\} \le2\e^{-t^2/2}. \] Consequently $\E\norm{\mathbf A}_{\rm op}^2\le4(M+N)+8$, and $\E\exp(c\norm{\mathbf A}_{\rm op}^2/N)\le3\exp(4c(M+N)/N)$ for $0t)\le2\e^{-t^2/2}$ and $\norm{\mathbf A}_{\rm op}^2\le4(M+N)+2T^2$. Then $\E T^2=\int_0^\infty2t\Prob(T>t)\dd t\le4$, which gives the second bound. For $0t)\dd t \le1+\frac{2s}{1/2-s}\le3, \] and the choice $s=2c/N$ gives the third bound. \end{proof} \paragraph{Poissonization.} We replace the number $M_N$ of rows by a Poisson variable $M\sim\mathrm{Poi}(\alpha N)$, and when passing to dimension $N'$ we add independently $\pi\sim\mathrm{Poi}(\alpha n)$ rows, so that the $N'$-dimensional system has $M+\pi\sim\mathrm{Poi}(\alpha N')$ rows; from now on the letter $\pi$ denotes this count. This changes the free energy by $o(1)$. To check this, let $\mathsf h_N(j)$ be the expected log partition function in dimension $N$ with $j$ rows. Adding a fresh row $g$ changes $\mathsf h_N$ by $\E\log\langle\e^{U(g\cdot\sigma/\sqrt N)}\rangle$, where $\langle\cdot\rangle$ is the Gibbs measure before the row is added. By Jensen's inequality and $U\le U_{\max}$, this lies between $\E U(G)$ and $U_{\max}$, because $g\cdot\sigma/\sqrt N$ is standard Gaussian for every fixed $\sigma$ and $g$ is independent of the Gibbs measure. By \eqref{I:lo:quadratic}, $\abs{\E U(G)}<\infty$. Hence $\mathsf h_N$ is Lipschitz with a constant independent of $N$, and \[ \abs{\E\mathsf h_N(M)-\mathsf h_N(M_N)}\le C\,\E\abs{M-M_N} \le C\bigl(\sqrt{\alpha N}+\abs{\alpha N-M_N}\bigr)=o(N). \] Poisson numbers of constraints are standard in the study of diluted spin glasses \cite{FranzLeone2003,PanchenkoTalagrand2004}. \paragraph{Conditioning on the number of rows.} The concentration estimates below are proved at a fixed number of rows, so we carry out the cavity computation conditionally on the value of $M$, for values in the window \[ W_N=\{j\in\N:\abs{j-\alpha N}\le N^{2/3}\}. \] The Poisson tail bound $\Prob(\abs{M-\alpha N}>t)\le2\exp(-t^2/(2(\alpha N+t)))$ gives $\Prob(M\notin W_N)\le2\e^{-cN^{1/3}}$, and for large $N$ every $j\in W_N$ satisfies $j\le2\alpha N$. Step~2 of \Cref{I:lo:proof} reduces the proof to one value of $M$ in $W_N$. From \Cref{I:lo:perturbations} on, $M$ denotes a fixed integer in $W_N$, expectations are conditional on this value, and all bounds and all terms $o(1)$ are uniform in $M\in W_N$; the exceptions are \Cref{I:lo-perturb}(i) and the quantities $\Xi_N$ of Step~1 of \Cref{I:lo:proof}, which concern the Poissonized systems. The number $\pi\sim\mathrm{Poi}(\alpha n)$ of rows added in dimension $N'$ remains Poisson throughout. \paragraph{The three systems.} We compare three systems. Each has $M$ rows, and the $N'$-dimensional system has $\pi$ further rows. \begin{enumerate}[label=\textup{(\alph*)}] \item The \emph{$N$-dimensional system} has configuration space $S_N$, rows $g^{(N)}_a\in\R^N$ for $a\le M$, and Hamiltonian $\sum_{a\le M}U(g^{(N)}_a\cdot\sigma/\sqrt N)$. \item The \emph{$N'$-dimensional system} has configuration space $S_{N'}$ and rows $(g_a,g'_a)\in\R^N\times\R^n$ for $a\le M$, together with $\pi$ further rows $\tilde g_i\in\R^{N'}$. We write $\langle\cdot\rangle_{N'}$ for its Gibbs measure with the first $M$ rows only. \item The \emph{reference system} has configuration space $S_N$, the rows $g_a\in\R^N$ for $a\le M$ (the first $N$ coordinates of the rows of the $N'$-dimensional system), and the fields \[ S_a(\sigma)=\frac{g_a\cdot\sigma}{\sqrt{N'}} . \] \end{enumerate} All rows are independent standard Gaussian vectors. We write $\bG$, $\bG'$ for the $M\times N$ and $M\times n$ matrices with rows $g_a$ and $g'_a$. In the coordinates $\rho=\rho(\sigma,\varepsilon)$ the fields of the $N'$-dimensional system are \begin{equation}\label{I:lo:D-fields} \frac{(g_a,g'_a)\cdot\rho}{\sqrt{N'}} =\chi(\varepsilon)S_a(\sigma)+w_a(\varepsilon),\qquad w_a(\varepsilon)=\frac{g'_a\cdot\varepsilon}{\sqrt{N'}} . \end{equation} The fields of the $N$-dimensional system can be realized from the same rows. Let $g''_a\in\R^N$ be independent standard Gaussian vectors, forming the matrix $\bG''$, and put \begin{equation}\label{I:lo:N-fields} \eta_a(\sigma)=c_N\,g''_a\cdot\sigma,\qquad c_N^2=\frac1N-\frac1{N'}=\frac{n}{NN'} . \end{equation} As processes indexed by $\sigma\in S_N$, the centered Gaussian families $(S_a+\eta_a)_{a\le M}$ and $(g^{(N)}_a\cdot\sigma/\sqrt N)_{a\le M}$ have the same covariance $\sigma\cdot\sigma'/N$, hence the same law. Both the $N'$-dimensional and the $N$-dimensional partition functions are therefore integrals against the reference system with modified fields. \paragraph{Observables of the reference system.} The following quantities determine the coordinate term: \begin{equation}\label{I:lo:observables} \begin{aligned} A_N(\sigma)&=\frac1N\sum_{a\le M}S_a(\sigma)U'(S_a(\sigma)),& B_N(\sigma)&=\frac1{N'}\sum_{a\le M}U''(S_a(\sigma)),\\ X_N(\sigma,\sigma')&=\frac1{N'}\sum_{a\le M}U'(S_a(\sigma))U'(S_a(\sigma')),& R_N(\sigma,\sigma')&=\frac{\sigma\cdot\sigma'}N,\\ b_N(\sigma)&=1+A_N(\sigma)-B_N(\sigma). \end{aligned} \end{equation} The spin overlap is $R_N$. We call $X_N$ the \emph{response overlap}; it is a Gram matrix, hence positive semidefinite, and its diagonal is $X_N(\sigma,\sigma)=N'^{-1}\sum_aU'(S_a(\sigma))^2$. The terms $A_N$ and $B_N$ arise below from the radial factor $\chi(\varepsilon)$ and from the second-order term of a Gaussian integration by parts, respectively, and $b_N-1$ will be the coefficient of $-(\norm\varepsilon^2-n)/2$ in the exponent of the Gaussian integral over $\varepsilon$. Let $\Omega_N$ be the event on which the operator norms of $\bG$, $\bG''$, and $\bG'$ are at most $C_1\sqrt N$. Since $M\le2\alpha N$, \Cref{I:lo:opnorm} with $t=\sqrt N$ shows that $\Prob(\Omega_N^c)\le6\e^{-N/2}$ for $N\ge n$, with the constant $C_1=\sqrt{2\alpha}+4$. Since $\sum_aS_a(\sigma)^2=\norm{\bG\sigma}^2/N'\le\norm\bG_{\rm op}^2$, the growth bounds in~\eqref{I:lo:class} give, for every $\sigma\in S_N$, \begin{equation}\label{I:lo:observable-bounds} \sum_{a\le M}S_a(\sigma)^2\le\norm\bG_{\rm op}^2,\qquad \abs{b_N(\sigma)}+X_N(\sigma,\sigma)+\abs{B_N(\sigma)} \le C\Big(1+\frac{\norm\bG_{\rm op}^2}N\Big). \end{equation} On $\Omega_N$ the right sides are at most $CN$ and $C$. By \Cref{I:lo:opnorm}, for every $c>0$ and $N\ge N_0(c)$, \begin{equation}\label{I:lo:expmoments} \E\exp\Big(c\sup_{\sigma\in S_N}\bigl(\abs{b_N(\sigma)} +X_N(\sigma,\sigma)\bigr)\Big)\le C_c , \end{equation} uniformly in $M\le2\alpha N$. In particular, all polynomial moments of these quantities and of $\norm\bG_{\rm op}^2/N$ are bounded. \section{The overlap representation theorem}\label{I:sec-overlap-input} The cavity computation requires a description of the limiting joint law of the spin overlap $R_N$ and the response overlap $X_N$. We use a general result about random probability measures on Hilbert spaces, which does not involve the perceptron Hamiltonian. In this section we state it, recall the three external results on which it rests, and derive it from them. \paragraph{Setting.} For each $N\ge1$, let $\mathfrak G_N$ be a random probability measure on the product of the unit balls of two separable Hilbert spaces, which may depend on $N$. For independent samples $(x_N^\ell,y_N^\ell)_{\ell\ge1}$ from $\mathfrak G_N$ put \[ R^{1,N}_{\ell\ell'}=x_N^\ell\cdot x_N^{\ell'},\qquad R^{2,N}_{\ell\ell'}=y_N^\ell\cdot y_N^{\ell'}\qquad(\ell,\ell'\ge1). \] We write $\E\langle\cdot\rangle_N$ for expectation over the random measure and its samples. The arrays take values in $[-1,1]$, they are positive semidefinite, and they are invariant in law under finite permutations of the sample labels. Suppose that the pair of arrays converges in finite-dimensional distribution to a pair $(R^1,R^2)$, and write $\E\langle\cdot\rangle$ for expectations of the limiting arrays. The limiting arrays have the same three properties. For $r\ge1$, rational $v_1,v_2\ge0$ and an integer $m\ge1$ put $Q(x,y)=(v_1x+v_2y)^m$. For a bounded measurable function $f$ of the arrays of the first $r$ samples, the Ghirlanda--Guerra discrepancy of \Cref{I:up:contact-sec}, written with $r$ replicas since $n$ is the cavity dimension here, is \begin{equation}\label{I:lo:GG-approx-defect} \begin{split} \mathfrak D^N_r(f,Q) &=\E\langle fQ(R^{1,N}_{1,r+1},R^{2,N}_{1,r+1})\rangle_N -\frac1r\E\langle f\rangle_N \E\langle Q(R^{1,N}_{12},R^{2,N}_{12})\rangle_N\\ &\quad-\frac1r\sum_{\ell=2}^r \E\langle fQ(R^{1,N}_{1\ell},R^{2,N}_{1\ell})\rangle_N . \end{split} \end{equation} Here $f$ may depend on the diagonal entries of the arrays. For a single random measure $\mathfrak G$ we drop the index $N$ and write $\mathfrak D_r(f,Q)$. \paragraph{Ruelle probability cascade arrays with paired levels.} We use the Ruelle probability cascade of \Cref{I:up:cascade-sec}: a depth $k\ge0$, parameters $0=\theta_0<\theta_1<\cdots<\theta_k<\theta_{k+1}=1$, masses $w_l=\theta_{l+1}-\theta_l$ for $0\le l\le k$, weights $(v_\beta)$ indexed by the leaves $\beta$ of the infinitely branching tree of depth $k$, and the depth $\beta\wedge\beta'$ of the last common ancestor of two leaves. If $\beta^1$ and $\beta^2$ are sampled independently from the weights $(v_\beta)$, then the averaged probability that $\beta^1\wedge\beta^2=l$ is $w_l$, by~\eqref{I:up-twopath} with the leaf function zero. Given levels $0\le q_0\le\cdots\le q_k$ and $0\le p_0\le\cdots\le p_k$, the \emph{Ruelle probability cascade array with paired levels $(q_l,p_l)$} is the pair of arrays with off-diagonal entries $(q_{\beta^\ell\wedge\beta^{\ell'}},p_{\beta^\ell\wedge\beta^{\ell'}})$, $\ell\ne\ell'$, where $\beta^1,\beta^2,\dots$ are independent samples from $(v_\beta)$. The law of its pair of $(1,2)$ entries is $\sum_{l=0}^kw_l\delta_{(q_l,p_l)}$. A single sequence of levels $0\le s_0<\cdots0$ there is $\delta>0$ with the following property. Let $\mathfrak G$ be a random probability measure on the product of the unit balls of two Hilbert spaces, with arrays $R^1,R^2$ as above. Suppose that $\abs{\mathfrak D_r(f,c_j)}\le\delta$ for all $r,j_1,j_2,j_3\le\lfloor\delta^{-1}\rfloor$ and all continuous functions $f$ of the arrays of $r$ samples with $\norm f_\infty\le1$, where $\mathfrak D_r$ is~\eqref{I:lo:GG-approx-defect} for $\mathfrak G$. Then, for every $\varphi\in C^\infty(\R^2)$, \[ \Bigl|\E\langle\varphi(R^1_{12},R^2_{12})\rangle -\E\varphi\bigl(F_1^{-1}(V),F_2^{-1}(V)\bigr)\Bigr| \le\epsilon\bigl(\norm\varphi_\infty+\norm{\nabla\varphi}_\infty\bigr), \] where $F_a^{-1}$ is the function $F_Y^{-1}$ for the law of $R^a_{12}$ under $\E\langle\cdot\rangle$. \item If $Y_{1,N}\to Y_1$ and $Y_{2,N}\to Y_2$ in law, then the monotone couplings converge in law: $(F_{Y_{1,N}}^{-1}(V),F_{Y_{2,N}}^{-1}(V))\to (F_{Y_1}^{-1}(V),F_{Y_2}^{-1}(V))$. \end{enumerate} \end{lemma} \begin{proof} Part~(a) is \cite[Theorem~5.3]{Mourrat2021}, whose hypothesis is formulated with the same defects, for continuous test functions of the arrays of $r$ samples including their diagonal entries. Part~(b) is \cite[Lemma~5.4]{Mourrat2021}. \end{proof} \begin{theorem}\label{I:thm-external-overlap} In the setting above, assume that for every $r\ge1$, every integer $m\ge1$, and all rational $v_1,v_2\ge0$, \begin{equation}\label{I:lo:GG-approx} \sup_{\norm f_\infty\le1}\abs{\mathfrak D^N_r(f,Q)}\longrightarrow0 \qquad(N\to\infty), \end{equation} where the supremum is over bounded measurable functions of the arrays of the first $r$ samples. Then the limiting pair satisfies the Ghirlanda--Guerra identities \begin{equation}\label{I:lo:GG} \E\langle fQ(R^1_{1,r+1},R^2_{1,r+1})\rangle =\frac1r\E\langle f\rangle\,\E\langle Q(R^1_{12},R^2_{12})\rangle +\frac1r\sum_{\ell=2}^r\E\langle fQ(R^1_{1\ell},R^2_{1\ell})\rangle \end{equation} for every bounded measurable $f$ of the arrays of its first $r$ samples. Moreover: \begin{enumerate}[label=\textup{(\roman*)}] \item The off-diagonal entries of $R^1$ and $R^2$ are nonnegative almost surely. \item The off-diagonal pair of arrays is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with paired nondecreasing levels $(q_l,p_l)\in[0,1]^2$. \end{enumerate} \end{theorem} Part~(ii) is the form, for a pair of Gram arrays, of Panchenko's synchronization combined with his approximation by Ruelle probability cascades \cite{Panchenko2015}. Step~1 of the proof follows the proof of \cite[Theorem~3]{Panchenko2015} in Section~3 there, Step~2 the positivity argument of \cite[Section~4]{Panchenko2015}, and Step~4 the approximation by cascades in \cite[Section~5]{Panchenko2015}. In Step~3 we use Mourrat's quantitative form, \Cref{I:lo:ext-sync}(a). It applies to the Gram pairs at finite $N$, before passing to the limit, so no representing measure of the limiting arrays needs to preserve their diagonal values. \begin{proof} We divide the proof into four steps. \proofstep{1}{The identities in the limit.} For continuous $f$, finite-dimensional convergence and~\eqref{I:lo:GG-approx} give~\eqref{I:lo:GG}, since all arrays take values in $[-1,1]$. Both sides of~\eqref{I:lo:GG} are integrals of $f$ against finite signed measures on the arrays of $r$ samples, and they agree on continuous functions, so they agree for bounded measurable $f$. Choosing $(v_1,v_2)=(1,0)$, $(0,1)$ and $(1/2,1/2)$ gives~\eqref{I:lo:GG-scalar} with $\psi(x)=x^m$, $m\ge1$, for the scalar arrays $R^1$, $R^2$, and $S/2$, where $S=R^1+R^2$, and for test functions $f$ of their off-diagonal entries. It also holds for constant $\psi$, since the sum of its three coefficients is one. By Weierstrass approximation it holds for continuous $\psi$ on $[-1,1]$, and, again by equality of finite measures, for bounded measurable $\psi$. \proofstep{2}{Positivity.} Each of the arrays $R^1$, $R^2$, and $S/2$ is a Gram--de Finetti array with diagonal entries at most one, as a limit of such arrays. By \Cref{I:lo:ext-DS}, its off-diagonal part is the overlap array of a random measure on a unit ball, and this measure satisfies the Ghirlanda--Guerra identities by Step~1. \Cref{I:lo:ext-GG}(a) gives~(i), and it shows that the law of the off-diagonal array $S/2$ is determined by the law of $S_{12}/2$. \proofstep{3}{Monotone coupling.} Let $\epsilon>0$, and let $\delta>0$ be given by \Cref{I:lo:ext-sync}(a). The hypothesis of \Cref{I:lo:ext-sync}(a) involves finitely many tuples $(r,j_1,j_2,j_3)$, and by~\eqref{I:lo:GG-approx} it holds for $\mathfrak G_N$ for all sufficiently large $N$. Hence, for $\varphi\in C^\infty(\R^2)$, \[ \Bigl|\E\langle\varphi(R^{1,N}_{12},R^{2,N}_{12})\rangle_N -\E\varphi\bigl(F_{1,N}^{-1}(V),F_{2,N}^{-1}(V)\bigr)\Bigr| \le\epsilon\bigl(\norm\varphi_\infty+\norm{\nabla\varphi}_\infty\bigr), \] where $F_{a,N}^{-1}$ is defined from the law of $R^{a,N}_{12}$. As $N\to\infty$, the first term converges to $\E\langle\varphi(R^1_{12},R^2_{12})\rangle$, and by \Cref{I:lo:ext-sync}(b) the second converges to $\E\varphi(F_1^{-1}(V),F_2^{-1}(V))$. Since $\epsilon$ is arbitrary, the law of $(R^1_{12},R^2_{12})$ is the monotone coupling of its marginals. The maps $F_1^{-1}$ and $F_2^{-1}$ are nondecreasing, so the image of $(0,1)$ under $v\mapsto(F_1^{-1}(v),F_2^{-1}(v))$ is totally ordered for the coordinatewise order, and so is its closure. Thus the support $\mathsf C$ of the law of $(R^1_{12},R^2_{12})$, a compact subset of $[0,1]^2$ by (i), is totally ordered. Two points $(x,y)$ and $(x',y')$ of $\mathsf C$ with $x+y\le x'+y'$ therefore satisfy \[ 0\le x'-x\le(x'+y')-(x+y),\qquad 0\le y'-y\le(x'+y')-(x+y). \] In particular the sum determines both coordinates: on the compact set $\{x+y:(x,y)\in\mathsf C\}$, which is the support of the law of $S_{12}$, there are nondecreasing $1$-Lipschitz functions $\mathsf Q_1,\mathsf Q_2$ with $(x,y)=(\mathsf Q_1(x+y),\mathsf Q_2(x+y))$ on $\mathsf C$. We extend them to nondecreasing $1$-Lipschitz functions on $\R$, by linear interpolation across the gaps of this compact set and by constants outside its hull. Almost surely $R^1_{12}=\mathsf Q_1(S_{12})$ and $R^2_{12}=\mathsf Q_2(S_{12})$. By exchangeability and a countable intersection, the same holds for every off-diagonal pair of indices. \proofstep{4}{Approximation by cascades.} Let $F^{-1}$ be the function $F_Y^{-1}$ for the law $\zeta_S$ of $S_{12}/2$, and let $\zeta_h$ be the law of $F^{-1}(\lceil hV\rceil/h)$. It has finite support contained in the support of $\zeta_S$, and it converges weakly to $\zeta_S$ as $h\to\infty$, since $0\le\lceil hV\rceil/h-V<1/h$ almost surely and $F^{-1}$ is continuous at almost every point of $(0,1)$. By Steps~1 and~2 and \Cref{I:lo:ext-GG}(b), the cascade arrays with overlap laws $\zeta_h$, with levels $s^h_0<\cdots0$, the quantities \[ \E\bigl\langle\exp\bigl(cX(\rho,\rho)+c\abs{\mathsf b(\rho)}\bigr) \bigr\rangle \] are bounded along the sequence from some index on, and suppose that the finite-dimensional distributions of the arrays converge. Then the following quantities converge, and their limits depend only on the limiting law of the arrays: \begin{enumerate}[label=\textup{(\roman*)}] \item $\E\log\mathcal N_\Lambda$ and $\E\log\mathcal D$; \item expectations under the cavity measure of bounded continuous functions of the arrays and the cavity coordinates of finitely many replicas; \item the expected logarithmic increment $\E\log\langle\exp\{\sum_{i\le\pi}U(y_i(\rho))\}\rangle$, for $\pi\sim\mathrm{Poi}(\alpha n)$ independent of the other variables, whenever $U$ is continuous and satisfies~\eqref{I:lo:quadratic}. \end{enumerate} In particular, if $\E\langle\abs{\mathsf b-\bar b_N}\rangle\to0$ for constants $\bar b_N\to b_\infty$, then $\mathsf b$ may be replaced by the constant $b_\infty$ in these limits. \end{lemma} \Cref{I:lo-cont} is the analogue for the perceptron of the continuity of the cavity functionals in the law of the overlap array, proved for mixed $p$-spin models in \cite[Lemma~3]{Panchenko2014Parisi} and for spherical models in \cite{Chen2013}. The difference is that the functionals here also depend on the response overlap and on the precision $\mathsf b$, and that part~(iii) treats the row term, whose activation may be unbounded below. The third lemma is an identity for the mean of $b_N$ under the reference system, obtained by Gaussian integration by parts in the rows $g_a$; it turns $S_aU'(S_a)$ into $U''(S_a)+U'(S_a)^2$ minus a two-replica term. In the limit it shows that the smallest precision of the Gaussian integral over the added coordinates is at least one (Step~3 of \Cref{I:lo:proof}). \begin{lemma}\label{I:lo-precision} Fix $n$. For two replicas $\sigma^1,\sigma^2$ of the reference system, uniformly in the perturbation parameters and in $M\in W_N$, \[ \E\langle b_N\rangle_{\rm ref} =1+\E\langle X_N(\sigma,\sigma)\rangle_{\rm ref} -\E\bigl\langle R_N(\sigma^1,\sigma^2)X_N(\sigma^1,\sigma^2) \bigr\rangle_{\rm ref}+o(1). \] \end{lemma} \paragraph{Finite cascades as random measures.} \Cref{I:lo-cont} is also applied to finite Ruelle probability cascades carrying prescribed diagonal values. Take a cascade as in \Cref{I:sec-overlap-input}, with depth $k$, weights $(v_\beta)$, and paired levels $0\le q_0\le\cdots\le q_k\le1$ and $0\le p_0\le\cdots\le p_k$, and let $\bar p\ge p_k$ and $\bar b\in\R$. Independently of $(v_\beta)$, let $Y^p$, $Y^c$, and $Y^{q,1},Y^{q,2},\dots$ be independent Gaussian cascade fields (\Cref{I:up:gaussian-sec}), with values in $\R^n$, $\R$, and $\R$, and with levels $p$, $c=(nq_lp_l)_{l\le k}$, and $q$, respectively; the levels $c_l$ are nonnegative and nondecreasing. The \emph{cascade realization} with diagonal values $(1,\bar p,\bar b)$ is the random measure $\mathfrak G=\sum_\beta v_\beta\,\delta_\beta\otimes\nu_n\otimes\nu_1 \otimes\nu_1^{\otimes\N}$ on $\mathsf E=\N_*^k\times\R^n\times\R\times\R^{\N}$, with $\mathsf b\equiv \bar b$, with kernels $R$ and $X$ equal to $q_{\beta\wedge\beta'}$ and $p_{\beta\wedge\beta'}$ at distinct points with leaves $\beta,\beta'$ and to $1$ and $\bar p$ at equal points, and with the fields \[ z(\rho)=Y^p_\beta+\sqrt{\bar p-p_k}\,x,\qquad y(\rho)=Y^c_\beta+\sqrt{n(\bar p-q_kp_k)}\,x',\qquad y_i(\rho)=Y^{q,i}_\beta+\sqrt{1-q_k}\,x''_i \] at $\rho=(\beta,x,x',x'')$. The within-state coordinates $x,x',x''$ carry the nonnegative differences between the diagonal values and the top levels. Two independent samples from $\mathfrak G$ are distinct points almost surely, so the arrays of the replicas are cascade arrays with paired levels $(q_l,p_l)$ off the diagonal, with diagonal values $1$, $\bar p$, and $\bar b$. Given the cascade weights and the leaves of the replicas, the values of the fields at the replicas are centered Gaussian with the covariances in~\ref{I:lo:G}, because the cascade fields are independent of the weights and the within-state coordinates are independent standard Gaussian. This gives~\ref{I:lo:G}, and the moment condition of \Cref{I:lo-cont} holds because $X(\rho,\rho)=\bar p$ and $\mathsf b=\bar b$ are constant. \paragraph{How the lemmas are combined with the overlap theorem.} Fix $M\in W_N$ and parameters in the set $\mathcal G_N(M)$ of \Cref{I:lo-perturb}(iii) (in \Cref{I:lo:proof}, $M=M_N$ depends on $N$), and pass to a subsequence along which the finite-dimensional distributions of the arrays $(R_N,X_N,b_N)$ under $\E\langle\cdot\rangle_{\rm ref}$ converge; they are tight by~\eqref{I:lo:observable-bounds} and~\eqref{I:lo:expmoments}. The reference system satisfies~\ref{I:lo:G} with $\mathfrak G=\langle\cdot\rangle_{\rm ref}$, $R=R_N$, $X=X_N$, and $\mathsf b=b_N$, for fields that are, conditionally on the disorder, independent Gaussian processes with the covariances in~\ref{I:lo:G}; for instance \begin{align*} z_i(\sigma)&=\sum_{a\le M}\frac{U'(S_a(\sigma))}{\sqrt{N'}}\,\mathsf g_{ai},\qquad y(\sigma)=\sqrt n\sum_{a\le M}\sum_{i\le N} \frac{\sigma_iU'(S_a(\sigma))}{\sqrt{NN'}}\,\mathsf g'_{ai},\\ y_j(\sigma)&=\sum_{i\le N}\frac{\sigma_i}{\sqrt N}\,\mathsf g''_{ji}, \end{align*} with independent standard Gaussian variables $\mathsf g_{ai},\mathsf g'_{ai},\mathsf g''_{ji}$, independent of the reference system. By~\eqref{I:lo:expmoments}, it satisfies the moment condition of \Cref{I:lo-cont}. We apply \Cref{I:thm-external-overlap} to the Gram pairs generated by $\sigma/\sqrt N$ and $\widehat\Psi_{N'}(\rho(\sigma,0))$. Their limiting arrays have $R^1=R$ and $R^2=X/C_X$, where $(R,X)$ is the limit of $(R_N,X_N)$, for three reasons. \begin{enumerate}[label=\textup{(\alph*)}] \item Both feature vectors lie in unit balls, and on $\Omega_N$ the response Gram array differs from $X_N/C_X$ by $O(n/N)$, by~\eqref{I:lo:reference-gram}; moreover $\Prob(\Omega_N^c)\to0$. \item The diagonals concentrate by \Cref{I:lo-perturb}(ii)--(iii), so the limiting diagonal values are $1$ and $p_\infty/C_X$, where \[ p_\infty=\lim\E\langle X_N(\sigma,\sigma)\rangle_{\rm ref} \] along a further subsequence. \item \Cref{I:lo-perturb}(iii) gives vanishing discrepancies $\mathfrak D^N_r(f,Q)$ of~\eqref{I:lo:GG-approx-defect}, uniformly over bounded tests, for each fixed number of replicas and each $Q=Q_{v,m}$, that is, for $v_1+v_2\le1$. Homogeneity of degree $m$ extends the conclusion to every nonnegative rational pair $(v_1,v_2)$, so~\eqref{I:lo:GG-approx} holds. \end{enumerate} By \Cref{I:thm-external-overlap}(ii), after multiplying the second levels of its approximating cascades by $C_X$, the off-diagonal limit of $(R_N,X_N)$ is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with paired nondecreasing levels $(q_l,p_l)$. The observable $b_N$ concentrates about $\E\langle b_N\rangle_{\rm ref}$, which converges along a further subsequence to a constant $b_\infty$; by the last assertion of \Cref{I:lo-cont}, $b_N$ may be replaced by $b_\infty$ in all limits. In the limit, $0\le R_{12}\le1$ and $0\le X_{12}\le p_\infty$ almost surely. Nonnegativity is \Cref{I:thm-external-overlap}(i), and $R_N\le1$. For the last bound, the Cauchy--Schwarz inequality gives $X_N(\sigma^1,\sigma^2)\le(X_N(\sigma^1,\sigma^1)+X_N(\sigma^2,\sigma^2))/2$, so $\E\langle(X_N(\sigma^1,\sigma^2)-p_\infty)_+\rangle_{\rm ref}\le \E\langle\abs{X_N(\sigma,\sigma)-p_\infty}\rangle_{\rm ref}\to0$, and the left side converges to the corresponding expectation in the limit by uniform integrability. Projecting the levels of the approximating cascades onto $[0,1]\times[0,p_\infty]$, which is monotone, continuous, and the identity on the values of the limiting arrays, we may assume $0\le q_l\le1$ and $0\le p_l\le p_\infty$. The cascade realizations of these cascades with diagonal values $(1,p_\infty,b_\infty)$ then have arrays converging to the limiting arrays of $(R_N,X_N,b_N)$. We apply \Cref{I:lo-cont} to the sequence that alternates the reference systems along our subsequence (with $b_N$ replaced by $b_\infty$) and these cascade realizations, whose arrays converge to the same limit. We conclude that the limits along our subsequence of the quantities in \Cref{I:lo-cont}(i)--(ii) for the reference system are the limits of their values on the cascade realizations. For the spin overlaps of the $N'$-dimensional system we apply \Cref{I:thm-external-overlap} to the Gram pairs generated by $\rho/\sqrt{N'}$ and the zero vector. Then every $Q$ in its hypothesis is $(v_1x)^m=v_1^mx^m$, so the hypothesis reduces to the defects with $v=(1,0)$ of \Cref{I:lo-perturb}(iii). By parts~(i)--(ii) of the theorem, the limiting spin overlap array is a limit in finite-dimensional distribution of Ruelle probability cascade arrays with nondecreasing levels in $[0,1]$, and the overlap laws of these cascades converge to the limiting overlap law $\tilde\zeta$ of the $N'$-dimensional system. \section{The two cavity terms}\label{I:lo:cavity-terms} We now compute the increment of the expected log partition function from dimension $N$ to dimension $N'$ at fixed perturbation parameters and a fixed number $M\in W_N$ of rows. Write $\log Z_{N'}^{M+\pi}$, $\log Z_{N'}^{M}$, and $\log Z_N^{M}$ for the log partition functions of the perturbed $N'$-dimensional system with all $M+\pi$ rows, of the same system with its first $M$ rows only, and of the perturbed $N$-dimensional system. The increment splits as \begin{equation}\label{I:lo:split} \E\log Z_{N'}^{M+\pi}-\E\log Z_N^{M} =\underbrace{\bigl(\E\log Z_{N'}^{M+\pi}-\E\log Z_{N'}^{M}\bigr)}_{\text{row term}} +\underbrace{\bigl(\E\log Z_{N'}^{M}-\E\log Z_N^{M}\bigr)}_{\text{coordinate term}} . \end{equation} \paragraph{The row term.} By \Cref{I:lo-perturb}(v), the row term is at least $\E\log\langle\exp\{\sum_{i\le\pi}U(y_i(\rho))\}\rangle_{N'}-o(1)$, where the fields $y_i$ of the added rows are centered Gaussian with covariance $\rho\cdot\rho'/N'$ and are independent of $\langle\cdot\rangle_{N'}$. Thus $\langle\cdot\rangle_{N'}$, with $R(\rho,\rho')=\rho\cdot\rho'/N'$, $X\equiv0$, $\mathsf b\equiv0$, and the fields $y_i$ of the added rows, satisfies~\ref{I:lo:G} and the moment condition of \Cref{I:lo-cont}. By \Cref{I:lo-cont}(iii), which applies by~\eqref{I:lo:quadratic}, the limit of this quantity along a subsequence along which the spin overlap array converges depends only on the limiting law of that array. For parameters in $\mathcal G_N(M)$ this array is a limit of Ruelle probability cascade arrays with nondecreasing levels in $[0,1]$ whose overlap laws $\zeta_h$ converge to the limiting overlap law $\tilde\zeta$ (\Cref{I:lo:perturbations}). Consider the $h$-th of these cascades, with levels $0\le q_0\le\cdots\le q_k\le1$ (we drop the index $h$ from its levels and masses), and its cascade realization with diagonal values $(1,0,0)$ and $p\equiv0$. Given $\pi$, the fields $y_i(\rho)=Y^{q,i}_\beta+\sqrt{1-q_k}\,x''_i$ of different rows are independent, so \[ \Big\langle\exp\Big\{\sum_{i\le\pi}U(y_i(\rho))\Big\}\Big\rangle =\sum_\beta v_\beta\prod_{i\le\pi} \E\bigl[\e^{U(Y^{q,i}_\beta+\sqrt{1-q_k}\,G)}\bigm|Y^{q,i}\bigr]. \] Apply \Cref{I:up-cascade} with the marks $(e^{(i)}_\nu)_{i\le\pi}$, where $e^{(i)}_\nu$ are the Gaussian increments of $Y^{q,i}$ at the node $\nu$ (with variance $q_0$ at the root and $q_l-q_{l-1}$ at depth $l$), and with the leaf function $\sum_{i\le\pi}\cT_{1,1-q_k}U(Y^{q,i}_\beta)$. The leaf function is bounded above and at least $-C\sum_i(1+(Y^{q,i}_\beta)^2)$ by~\eqref{I:lo:quadratic} and~\eqref{I:pf:quadlower}, so all quantities in~\eqref{I:up-recursion} are finite. Since the marks of different rows are independent and the leaf function is a sum over the rows, the recursion~\eqref{I:up-recursion} factorizes over the rows. For one row, its steps are $\cT_{\theta_l,q_l-q_{l-1}}$ at depth $l$, and the average over the root mark is $\cT_{0,q_0}$. By~\eqref{I:up-stableidentity} and the recursion~\eqref{I:pf:recursion}, the expected logarithm of the display equals \[ \pi\,\bigl[\cT_{0,q_0}\cT_{\theta_1,q_1-q_0}\cdots\cT_{\theta_k,q_k-q_{k-1}} \cT_{1,1-q_k}U\bigr](0)=\pi\,u_{\zeta_h}(0,0;U), \] where $\zeta_h=\sum_lw_l\delta_{q_l}$ is the overlap law of the cascade, a probability measure on $[0,1]$ whose distribution function equals $0$ on $[0,q_0)$ and $\theta_{l+1}$ on $[q_l,q_{l+1})$ for $0\le l\le k$, with $q_{k+1}=1$ (some of these intervals may be empty); its steps are exactly the operators in the display. Averaging over $\pi\sim\mathrm{Poi}(\alpha n)$ gives $\alpha n\,u_{\zeta_h}(0,0;U)$. As $h\to\infty$, these values converge, by \Cref{I:lo-cont}(iii) applied to the sequence that alternates the $N'$-dimensional systems and the cascade realizations, to the limit of the lower bound for the row term. Moreover $u_{\zeta_h}(0,0;U)\to u_{\tilde\zeta}(0,0;U)$ by \Cref{I:lo-phi-cont}, since the laws $\zeta_h$ converge weakly on $[0,1]$ and hence in the Wasserstein distance. Thus, along the subsequences used below, \begin{equation}\label{I:lo:row-term} \liminf_N\,(\text{row term})\ge\alpha n\,u_{\tilde\zeta}(0,0;U). \end{equation} In \Cref{I:lo:proof} we show that $\tilde\zeta$ equals the limiting overlap law $\zeta$ of the reference system. \paragraph{Reduction of the coordinate term to the reference system.} By~\eqref{I:lo-density} and~\eqref{I:lo:D-fields}, $Z_{N'}^M$ is an integral over $(\sigma,\varepsilon)$ against $\mu_N(\dd\sigma)f_{N,n}(\varepsilon)\dd\varepsilon$. Restricting $\varepsilon$ to the ball $\mathbb B_\Lambda$ decreases it, so the coordinate term is bounded below by the same expression with $Z_{N'}^M$ replaced by its restriction to $\mathbb B_\Lambda$. By \Cref{I:lo-perturb}(iv)(a) we may replace the Gaussian part of the perturbation at $\rho(\sigma,\varepsilon)$ by its value at $\rho(\sigma,0)$, at a cost $o(1)$. After this replacement the Gaussian part depends only on $\sigma$. We add and subtract the activation terms $\sum_aU^\xi(S_a)$ at the reference fields, with $\xi=\xi_{N'}$, which include the scalar part, and factor out the partition function of the tilted reference system. Similarly, by \Cref{I:lo-perturb}(iv)(b) the $N$-dimensional system may be written, at a cost $o(1)$, with the fields $S_a+\eta_a$ of~\eqref{I:lo:N-fields}, the reference perturbation, and the strength $\xi_{N'}$. Since the reference partition function is common to both terms, it cancels, and the coordinate term is at least \begin{multline}\label{I:lo:before-interpolation} \E\log\Big\langle\int_{\mathbb B_\Lambda}f_{N,n}(\varepsilon) \e^{\sum_a[U^\xi(\chi(\varepsilon)S_a+w_a(\varepsilon))-U^\xi(S_a)]} \dd\varepsilon\;\e^{-nB_N/2}\Big\rangle_{\rm ref}\\ -\E\log\Big\langle\e^{\sum_a[U^\xi(S_a+\eta_a)-U^\xi(S_a)]} \e^{-nB_N/2}\Big\rangle_{\rm ref}-o(1). \end{multline} The factors $\e^{-nB_N/2}$ undo the tilt of $\langle\cdot\rangle_{\rm ref}$; they appear in both terms. Here $S_a=S_a(\sigma)$, $B_N=B_N(\sigma)$, and $\langle\cdot\rangle_{\rm ref}$ acts on $\sigma$. \paragraph{Gaussian interpolation for the numerator.} We replace the activation increment in the first term of~\eqref{I:lo:before-interpolation} by a Gaussian field. Let $z(\sigma)=(z_1(\sigma),\dots,z_n(\sigma))$ be, conditionally on the disorder, a centered Gaussian process with $\E z_i(\sigma)z_j(\sigma')=\delta_{ij}X_N(\sigma,\sigma')$, independent of everything else (for instance the process $z$ of \Cref{I:lo:perturbations}). For $0\le t\le1$ put $S_a^t=\chi(\varepsilon)S_a+\sqrt t\,w_a(\varepsilon)$ and \[ \Delta_t=\sum_a\bigl[U^\xi(S_a^t)-U^\xi(S_a)\bigr] +\sqrt{1-t}\,\varepsilon\cdot z(\sigma) +(1-t)\frac{\norm\varepsilon^2}{2}B_N(\sigma), \] and let $\varphi(t)$ be the first term of~\eqref{I:lo:before-interpolation} with the exponent replaced by $\Delta_t$; thus $\varphi(1)$ is that term. Write $\langle\cdot\rangle_t$ for the associated Gibbs measure on $S_N\times\mathbb B_\Lambda$. We first discuss the terms produced by the unperturbed activation $U$ and then the terms that carry a factor $\xi$ or $\xi^2$. The field $w_a(\varepsilon)$ is centered Gaussian with $\E w_a(\varepsilon)w_a(\varepsilon')=\varepsilon\cdot\varepsilon'/N'$, independently over $a$, and it is independent of the reference system. Gaussian integration by parts in the fresh columns $g'_a$ produces the diagonal terms $\norm\varepsilon^2(U''+U'^2)(S_a^t)$ (the $U''$ part from differentiating $U'$, the $U'^2$ part from differentiating the Gibbs density of the same replica) and a two-replica term. Integration by parts in $z$ produces the $U'^2$ part and the two-replica term at the reference fields $S_a$, with the opposite sign, and the derivative of the explicit term $(1-t)\norm\varepsilon^2B_N/2$ supplies the $U''$ part at the reference fields, also with the opposite sign. The result, for the activation $U$, is \begin{equation}\label{I:lo:interp-derivative} \begin{aligned} \varphi'(t)={}&\frac1{2N'}\sum_a\E\bigl\langle\norm\varepsilon^2 \bigl[(U''+U'^2)(S_a^t)-(U''+U'^2)(S_a)\bigr]\bigr\rangle_t\\ &-\frac1{2N'}\sum_a\E\bigl\langle(\varepsilon^1\cdot\varepsilon^2) \bigl[U'(S_a^{t,1})U'(S_a^{t,2})-U'(S_a^1)U'(S_a^2)\bigr]\bigr\rangle_t , \end{aligned} \end{equation} where the superscripts $1,2$ refer to two replicas $(\sigma^1,\varepsilon^1)$, $(\sigma^2,\varepsilon^2)$. The first line collects the diagonal contributions and the second the two-replica covariance contributions. We bound~\eqref{I:lo:interp-derivative} on $\Omega_N$. Uniformly on $S_N\times\mathbb B_\Lambda$, \[ \sum_aS_a^2\le CN,\qquad \sum_a\abs{S_a^t-S_a}^2\le C_{n,\Lambda}. \] The second bound follows from $S_a^t-S_a=(\chi(\varepsilon)-1)S_a+\sqrt t\,w_a(\varepsilon)$, from $\chi(\varepsilon)-1=O_{n,\Lambda}(N^{-1})$, and from $\sum_aw_a(\varepsilon)^2\le\norm{\bG'}_{\rm op}^2\norm\varepsilon^2/N' \le C\norm\varepsilon^2$. Since $\abs{U'}$ grows at most linearly and $U'',U'''$ are bounded, the function $\omega_U=U''+U'^2$ satisfies $\abs{\omega_U(x)-\omega_U(x')}\le C(1+\abs x+\abs{x-x'})\abs{x-x'}$, and the Cauchy--Schwarz inequality gives \begin{align*} \frac1{N'}\sum_a\abs{\omega_U(S_a^t)-\omega_U(S_a)} &\le\frac C{N'}\Big(\sum_a(1+\abs{S_a}+\abs{S_a^t-S_a})^2\Big)^{1/2} \Big(\sum_a\abs{S_a^t-S_a}^2\Big)^{1/2}\\ &\le C_{n,\Lambda}N^{-1/2}. \end{align*} The product difference in the second line obeys the same estimate, after writing it as $[U'(S_a^{t,1})-U'(S_a^1)]U'(S_a^{t,2})+U'(S_a^1) [U'(S_a^{t,2})-U'(S_a^2)]$. Since $\norm\varepsilon^2\le\Lambda n$ and $\abs{\varepsilon^1\cdot\varepsilon^2}\le\Lambda n$ on $\mathbb B_\Lambda$, the right side of~\eqref{I:lo:interp-derivative} is $O_{n,\Lambda}(N^{-1/2})$ on $\Omega_N$. On $\Omega_N^c$ the integrand is bounded by a polynomial in the operator norms, whose moments are bounded, so its contribution is at most $C_{n,\Lambda}\Prob(\Omega_N^c)^{1/2}$. The terms carrying $\xi$ or $\xi^2$ arise because the interpolation uses the activation $U^\xi$ of~\eqref{I:lo:U-xi} while $B_N$ and $X_N$ are defined with $U$. In~\eqref{I:lo:interp-derivative} they replace $\omega_U(S_a^t)$ by $\omega_{U^\xi}(S_a^t)$ and $U'(S_a^{t,1})U'(S_a^{t,2})$ by $(U^\xi)'(S_a^{t,1})(U^\xi)'(S_a^{t,2})$. By~\eqref{I:lo:class} the first and second derivatives of $\tau_1$ and $\tau_2$ grow at most linearly, and the first derivatives of $U$ as well; hence $\abs{\omega_{U^\xi}(x)-\omega_U(x)}\le C(\xi+\xi^2)(1+x^2)$ and $\abs{(U^\xi)'(x)(U^\xi)'(x')-U'(x)U'(x')}\le C(\xi+\xi^2)(1+x^2+x'^2)$. Since $N'^{-1}\sum_a(1+(S_a^t)^2)\le C$ on $\Omega_N$, these terms contribute $O_{n,\Lambda}(\xi_{N'}+\xi_{N'}^2)=o(1)$ on $\Omega_N$; this is sufficient, although it is weaker than the rate $N^{-1/2}$ available for the activation $U$. We conclude that $\varphi(1)-\varphi(0)=o(1)$ for fixed $n$ and $\Lambda$. The same holds for Gibbs expectations: differentiating $\E\langle F\rangle_t$ for a bounded measurable function $F$ of the spin overlaps and cavity coordinates of $r$ replicas, which does not depend on the fresh Gaussian variables, inserts at most $r+2$ replicas and the same covariance differences, so $\abs{\partial_t\E\langle F\rangle_t}\le C_{r,n,\Lambda}\norm F_\infty(N^{-1/2}+\xi_{N'}+\xi_{N'}^2)$ on $\Omega_N$, up to a contribution $C_{r,n,\Lambda}\norm F_\infty\Prob(\Omega_N^c)^{1/2}$. \paragraph{The numerator at $t=0$.} By Taylor's formula and~\eqref{I:lo:density-expansion}, uniformly on $\mathbb B_\Lambda$ and on $\Omega_N$, \[ \begin{split} \sum_a\bigl[U(\chi(\varepsilon)S_a)-U(S_a)\bigr] &=(\chi(\varepsilon)-1)\sum_aS_aU'(S_a)+O\Big((\chi(\varepsilon)-1)^2 \sum_aS_a^2\Big)\\ &=\frac{n-\norm\varepsilon^2}{2}A_N+O_{n,\Lambda}(N^{-1}), \end{split} \] and the corresponding terms carrying $\xi$ are $O_{n,\Lambda}(\xi_{N'})$. Adding $\norm\varepsilon^2B_N/2$ and using $b_N-1=A_N-B_N$, the exponent $\Delta_0$ equals \[ \varepsilon\cdot z(\sigma)-\frac{\norm\varepsilon^2-n}{2}\,(b_N-1) +\frac n2B_N+o(1). \] The term $nB_N/2$ cancels the factor $\e^{-nB_N/2}$ in~\eqref{I:lo:before-interpolation}, and $f_{N,n}(\varepsilon)\dd\varepsilon$ may be replaced by $\nu_n(\dd\varepsilon)$ by~\eqref{I:lo:density-expansion}. This explains the definitions of $A_N$, $B_N$, and $b_N$. \paragraph{Gaussian interpolation for the denominator.} The second term of~\eqref{I:lo:before-interpolation} is treated by the same interpolation. The field $\eta_a(\sigma)=c_Ng''_a\cdot\sigma$ has covariance $c_N^2\,\sigma\cdot\sigma'=(n/N')R_N(\sigma,\sigma')$, and on $\Omega_N$ we have $\sum_a\eta_a^2\le c_N^2\norm{\bG''}_{\rm op}^2N\le C_n$. Let $y(\sigma)$ be, conditionally on the disorder, a centered Gaussian process with covariance $nR_N(\sigma,\sigma')X_N(\sigma,\sigma')$, and interpolate with \[ \sum_a\bigl[U^\xi(S_a+\sqrt t\,\eta_a)-U^\xi(S_a)\bigr] +\sqrt{1-t}\,y(\sigma)+(1-t)\frac n2B_N(\sigma). \] Integration by parts gives~\eqref{I:lo:interp-derivative} with $\norm\varepsilon^2$ replaced by $n$ and $\varepsilon^1\cdot\varepsilon^2$ replaced by $nR_N(\sigma^1,\sigma^2)$, and the same estimates show that the two endpoints differ by $o(1)$. At $t=0$ the activation increment is replaced by $y(\sigma)+nB_N(\sigma)/2$, and $nB_N/2$ again cancels the factor $\e^{-nB_N/2}$. \paragraph{The coordinate term.} Combining the preceding paragraphs, for fixed $n$ and $\Lambda$ the coordinate term is at least \begin{equation}\label{I:lo:coordinate-N} \E\log\Big\langle\int_{\mathbb B_\Lambda}\nu_n(\dd\varepsilon)\, \e^{\varepsilon\cdot z(\sigma)-(\norm\varepsilon^2-n)(b_N(\sigma)-1)/2} \Big\rangle_{\rm ref} -\E\log\bigl\langle\e^{y(\sigma)}\bigr\rangle_{\rm ref}-o(1). \end{equation} The same comparison holds for Gibbs expectations of bounded continuous functions of finitely many replicas, including their cavity coordinates: under the Gibbs measure of the $N'$-dimensional system with $M$ rows, conditioned on $\varepsilon\in\mathbb B_\Lambda$ for each replica, such expectations differ by $o(1)$ from those under the cavity measure of the first term of~\eqref{I:lo:coordinate-N}, where the spin overlaps are $R_N(\sigma^\ell,\sigma^{\ell'})$. Indeed, the chain of comparisons above consists of \Cref{I:lo-perturb}(iv)(a), the interpolation, whose effect on Gibbs expectations was bounded above, and the replacement of an exponent and a density by quantities that differ from them by $o(1)$ uniformly on $\Omega_N\times\mathbb B_\Lambda$. Moreover, on $\mathbb B_\Lambda$ the spin overlap $\rho^\ell\cdot\rho^{\ell'}/N' =[\chi(\varepsilon^\ell)\chi(\varepsilon^{\ell'})N R_N(\sigma^\ell,\sigma^{\ell'})+\varepsilon^\ell\cdot\varepsilon^{\ell'}]/N'$ of the $N'$-dimensional system differs from $R_N(\sigma^\ell,\sigma^{\ell'})$ by $O_{n,\Lambda}(N^{-1})$, and a continuous function is uniformly continuous on the compact set of overlaps in $[-1,1]$ and cavity coordinates in $\mathbb B_\Lambda$. For parameters in $\mathcal G_N(M)$ and along the subsequences of \Cref{I:lo:perturbations}, \Cref{I:lo-cont} allows us to replace $b_N$ by its limit $b_\infty$. Thus, for fixed $n$ and $\Lambda$, the coordinate term is bounded below, up to $o(1)$, by \begin{equation}\label{I:lo-coordinate} \E\log\Big\langle\int_{\mathbb B_\Lambda}\nu_n(\dd\varepsilon)\, \e^{\varepsilon\cdot z(\sigma)-(\norm\varepsilon^2-n)(b_\infty-1)/2} \Big\rangle_{\rm ref} -\E\log\bigl\langle\e^{y(\sigma)}\bigr\rangle_{\rm ref}, \end{equation} where, conditionally on the reference system, $z$ and $y$ are centered Gaussian with \[ \E z_i(\sigma)z_j(\sigma')=\delta_{ij}X_N(\sigma,\sigma'),\qquad \E y(\sigma)y(\sigma')=nR_N(\sigma,\sigma')X_N(\sigma,\sigma'). \] The first term of~\eqref{I:lo-coordinate} is the expected logarithm of the numerator $\mathcal N_\Lambda$ of \Cref{I:lo-cont} for the reference system with $\mathsf b=b_\infty$, and the second is minus that of the denominator $\mathcal D$; we call the two expected logarithms the numerator and the denominator. By \Cref{I:lo-cont}(i), both converge along the subsequence, and by \Cref{I:lo:perturbations} their limits are the limits of their values on cascade realizations. We compute these values next. \section{The cavity quantities on cascades}\label{I:lo:cascades} We evaluate the numerator and the denominator of~\eqref{I:lo-coordinate} on a cascade realization, using \Cref{I:gc:finite}, and we show that the cutoff $\Lambda$ can be removed when the smallest precision is bounded below. Throughout this section we use the notation of \Cref{I:up:gaussian-sec}: a Ruelle probability cascade of depth $k$ with parameters $\theta_l$ and masses $w_l$, the branching level $J=\beta^1\wedge\beta^2$ of two replicas, and, for levels $p\in\mathcal K$ and a precision $b$, the precisions $d_l$, the mean levels $m_l$, the function $\cG(b,p)$, the mean squared norm $\varrho=m_k+1/b$, and the tilted measure $\cM$ defined in~\eqref{I:gc:parameters} and below it. Fix a cascade with paired levels $0\le q_0\le\cdots\le q_k\le1$ and $0\le p_0\le\cdots\le p_k$, and its cascade realization with diagonal values $(1,\bar p,\bar b)$ (\Cref{I:lo:perturbations}). The overlap law of the cascade is $\sum_lw_l\delta_{q_l}$, and the law of the response overlap is $\sum_lw_l\delta_{p_l}$. On the realization, the response field $z$ is the cascade field $Y^p_\beta$ plus the within-state variable $\sqrt{\bar p-p_k}\,x$. Averaging $x$ inside the Gibbs average multiplies the integrand of the numerator by $\e^{(\bar p-p_k)\norm\varepsilon^2/2}$, which changes the coefficient of $-\norm\varepsilon^2/2$ from $\bar b-1$ to $b-1$, where \begin{equation}\label{I:lo:b} b=\bar b-(\bar p-p_k). \end{equation} By~\eqref{I:gc:sbp}, the smallest precision $d_0=b-\sum_{l\ge1}\theta_l\Delta p_l$ of~\eqref{I:gc:parameters} is expressed through the mean of the response overlap: \begin{equation}\label{I:lo:d0} d_0=b-p_k+\sum_{l=0}^kw_lp_l . \end{equation} When $d_0>0$, all precisions are positive and \Cref{I:gc:finite} applies. \paragraph{The cavity quantities on a cascade realization.} Averaging the within-state coordinates inside the Gibbs average gives the following values. The numerator of~\eqref{I:lo-coordinate}, with $b_\infty$ replaced by $\bar b$, equals $\Theta_\Lambda+n(\bar b-1)/2$, where \[ \Theta_\Lambda=\E\log\sum_\beta v_\beta\int_{\mathbb B_\Lambda}\nu_n(\dd\varepsilon) \e^{\varepsilon\cdot Y^p_\beta-(b-1)\norm\varepsilon^2/2}, \] and we write $\Theta$ for the same quantity with $\varepsilon$ integrated over $\R^n$. When $d_0>0$, \eqref{I:gc:value} gives $\Theta=n[\cG(b,p)-\frac12\log b]$. The denominator equals \[ \E\log\sum_\beta v_\beta\e^{Y^c_\beta+n(\bar p-q_kp_k)/2} =\frac n2\Big(\bar p-\sum_{l=0}^kw_lq_lp_l\Big), \] by~\eqref{I:gc:linear} with the levels $c_l=nq_lp_l$; here $\bar p\ge p_k\ge q_kp_k$. Hence, when $d_0>0$, the unrestricted numerator minus the denominator, divided by $n$, is \begin{equation}\label{I:lo-v} \cG(b,p)-\frac12\log b +\frac12\Big(\bar b-1-\bar p+\sum_{l=0}^kw_lq_lp_l\Big). \end{equation} The last term vanishes in the limit by \Cref{I:lo-precision} (Step~3 of \Cref{I:lo:proof}). The cutoff is removed with the following lemma. It concerns the unrestricted tilted measure $\cM$ of \Cref{I:up:gaussian-sec} with the field $Y=Y^p$, defined when $d_0>0$, and the random probability measure $\cM_\Lambda$ on pairs $(\beta,\varepsilon)$ with density proportional to $v_\beta\nu_n(\dd\varepsilon)\e^{\varepsilon\cdot Y_\beta-(b-1)\norm\varepsilon^2/2}\1_{\mathbb B_\Lambda}(\varepsilon)$, which is defined for every $b$; when $d_0>0$, $\cM_\Lambda$ is $\cM$ conditioned on $\{\varepsilon\in\mathbb B_\Lambda\}$. \begin{lemma}\label{I:lo-positive-cont} Fix the cavity dimension $n$ and constants $P_0,B_0,c_0>0$, and consider finite cascades with \[ 0\le p_k\le P_0,\qquad d_0\ge c_0,\qquad b\le B_0 . \] There are constants $C,c'>0$, depending only on $n,P_0,B_0,c_0$, such that for all $\Lambda\ge1$ \begin{equation}\label{I:lo-uniform-cutoff} 0\le\Theta-\Theta_\Lambda\le C\e^{-c'\Lambda}, \qquad \Prob_\cM(\varepsilon^1\notin\mathbb B_\Lambda)\le C\e^{-c'\Lambda} . \end{equation} Moreover $\varrho=m_k+1/b\le P_0/c_0^2+1/c_0$. \end{lemma} \begin{proof} Every denominator $d_{l-1}d_l$ and $d_0^2$ in~\eqref{I:gc:parameters} is at least $c_0^2$, and the increments of $p$ sum to $p_k\le P_0$, so $m_k\le P_0/c_0^2$; also $1/b\le1/d_0\le1/c_0$. By~\eqref{I:gc:law}, one replica $\varepsilon^1$ has the law $N(0,\varrho I_n)$ under $\Prob_\cM$, and the Gaussian tail bound for $\norm{\varepsilon^1}^2$ gives the second estimate in~\eqref{I:lo-uniform-cutoff}, for $\Lambda$ larger than a constant; enlarging $C$ covers the remaining $\Lambda\ge1$. A small averaged probability of $\{\varepsilon\notin\mathbb B_\Lambda\}$ alone would not bound $\Theta-\Theta_\Lambda$, so we use convexity. Define the leaf functions \begin{align*} \mathsf f(y)&=\log\int\nu_n(\dd\varepsilon) \e^{\varepsilon\cdot y-(b-1)\norm\varepsilon^2/2},\\ \mathsf f_\Lambda(y)&=\log\int_{\mathbb B_\Lambda}\nu_n(\dd\varepsilon) \e^{\varepsilon\cdot y-(b-1)\norm\varepsilon^2/2}\qquad(y\in\R^n). \end{align*} Then $\mathsf f_\Lambda\le\mathsf f$, so $\Theta_\Lambda\le\Theta$. The map $\varphi\mapsto\E\log\sum_\beta v_\beta\e^{\varphi(Y_\beta)}$ is convex on leaf functions, and for a bounded direction $\psi$ its derivative at $\mathsf f$ is $\E_\cM\psi(Y_{\beta^1})$, by dominated convergence. Applying convexity with the bounded direction $\psi_t=\max\{\mathsf f_\Lambda,\mathsf f-t\}-\mathsf f\in[-t,0]$ gives \[ \Theta-\E\log\sum_\beta v_\beta\e^{\max\{\mathsf f_\Lambda,\mathsf f-t\}(Y_\beta)} \le\E_\cM\bigl[\mathsf f(Y_{\beta^1})-\mathsf f_\Lambda(Y_{\beta^1})\bigr], \] and letting $t\to\infty$, by monotone convergence, $\Theta-\Theta_\Lambda\le\E_\cM[\mathsf f(Y_{\beta^1})-\mathsf f_\Lambda(Y_{\beta^1})]$. Now $\exp(\mathsf f_\Lambda-\mathsf f)(y)$ is the probability that a vector with law $N(y/b,b^{-1}I_n)$ lies in $\mathbb B_\Lambda$, and by \Cref{I:gc:finite}(ii), $Y_{\beta^1}/b=M_k\sim N(0,m_kI_n)$ under $\Prob_\cM$. Hence \[ \Theta-\Theta_\Lambda\le \E\Big[-\log\Prob\bigl\{M_k+b^{-1/2}G'\in\mathbb B_\Lambda \bigm|M_k\bigr\}\Big], \] where $G'$ is a standard Gaussian vector in $\R^n$ independent of $M_k\sim N(0,m_kI_n)$. We split the last expectation according to the size of $M_k$. On $\{\norm{M_k}\le\sqrt{\Lambda n}/2\}$, the conditional probability of leaving $\mathbb B_\Lambda$ is at most $\Prob(\norm{G'}^2>b\Lambda n/4)\le\Prob(\norm{G'}^2>c_0\Lambda n/4)\le C\e^{-c'\Lambda}$, and for $\Lambda$ larger than a constant the negative logarithm is at most twice this bound, since $-\log(1-x)\le2x$ for $0\le x\le1/2$. For all $M_k$ and $\Lambda\ge1$, integrating the Gaussian density of $M_k+b^{-1/2}G'$ over the fixed ball $\mathbb B_1$ and using $\norm{x-M_k}^2\le2\norm x^2+2\norm{M_k}^2$ gives \[ \Prob\bigl\{M_k+b^{-1/2}G'\in\mathbb B_\Lambda\bigm|M_k\bigr\} \ge c_2\exp\bigl[-C_2(1+\norm{M_k}^2)\bigr], \] with $c_2,C_2$ depending only on $n,c_0,B_0$, since $c_0\le b\le B_0$. The contribution of $\{\norm{M_k}>\sqrt{\Lambda n}/2\}$ is therefore bounded by a Gaussian tail expectation of $\log(1/c_2)+C_2(1+\norm{M_k}^2)$, which is also at most $C\e^{-c'\Lambda}$ uniformly for $m_k\le P_0/c_0^2$. Enlarging $C$ covers the remaining $\Lambda\ge1$. \end{proof} The measure $\cM_\Lambda$ is the law of the leaves and cavity coordinates under the cavity measure of the cascade realization on $\mathbb B_\Lambda$, after the within-state coordinates are integrated out. For $r$ replicas, $\cM_\Lambda$ is $\cM$ conditioned on the event that $\varepsilon^\ell\in\mathbb B_\Lambda$ for $\ell\le r$, realization by realization. If a probability measure gives mass $1-\delta$ to an event, then conditioning each of $r$ replicas on this event changes the expectation of a bounded function $F$ of the $r$ replicas by at most $2r\norm F_\infty\delta$, because the conditioning event for the $r$ replicas has mass at least $1-r\delta$. Since this bound is linear in $\delta$, it survives averaging over the disorder: \begin{equation}\label{I:lo:conditioning} \abs{\E_{\cM_\Lambda}F-\E_\cM F}\le2r\norm F_\infty \Prob_\cM(\varepsilon^1\notin\mathbb B_\Lambda). \end{equation} \section{Rotation invariance}\label{I:lo:rotation-sec} The two identities that determine the entropy come from the following lemma. It uses only the invariance~\eqref{I:lo:invariance} of the $N'$-dimensional system under simultaneous rotations of the configurations and the rows. In the coordinates $\rho=\rho(\sigma,\varepsilon)$, the quantity $R^{\rm cav}$ below is the overlap $\varepsilon^1\cdot\varepsilon^2/n$ of the added coordinates. \begin{lemma}\label{I:lo-rotation} Suppose that the disorder-averaged law of a random Gibbs measure on $S_{N'}$ is invariant under simultaneous orthogonal transformations of its replicas. For two replicas $\rho^1,\rho^2$ define \[ R=\frac{\rho^1\cdot\rho^2}{N'},\qquad R^{\rm cav}=\frac1n\sum_{i=N+1}^{N'}\rho_i^1\rho_i^2 . \] Then \begin{equation}\label{I:lo-rotation-bound} \E\langle(R^{\rm cav}-R)^2\rangle\le\frac{3N'}{n(N'-1)} . \end{equation} Moreover, the averaged one-replica law is uniform on $S_{N'}$, and the law of its last $n$ coordinates converges to $\nu_n$ as $N\to\infty$. \end{lemma} \begin{proof} By the invariance, we may replace two given replicas by their images under an independent Haar-distributed rotation without changing the averaged law. Condition on two unit vectors $\rho^1/\sqrt{N'}$ and $\rho^2/\sqrt{N'}$ with inner product $R$. Their images can be written as $x$ and $Rx+\sqrt{1-R^2}\,x^\perp$, where $x$ is uniform on the unit sphere of $\R^{N'}$ and, given $x$, $x^\perp$ is uniform on the unit sphere of the orthogonal complement of $x$. For the $n$ coordinate indices $i\in\{N+1,\dots,N'\}$ put $A=\sum_ix_i^2$ and $B=\sum_ix_ix_i^\perp$, so that $R^{\rm cav}=(N'/n)(RA+\sqrt{1-R^2}\,B)$. The variable $A$ has the beta distribution with parameters $n/2$ and $N/2$, so \[ \E A=\frac n{N'},\qquad \Var(A)=\frac{2nN}{N'^2(N'+2)}\le\frac{2n}{N'(N'-1)} . \] Given $x$, $\E[x^\perp_ix^\perp_j\mid x]=(\delta_{ij}-x_ix_j)/(N'-1)$, hence $\E[B\mid x]=0$ and $\E[B^2\mid x]=(A-A^2)/(N'-1)$. Therefore \[ \E B^2\le\frac n{N'(N'-1)},\qquad \E[(A-n/N')B]=0 . \] Consequently \[ \E\bigl[(RA+\sqrt{1-R^2}\,B-Rn/N')^2\bigr] =R^2\Var(A)+(1-R^2)\E B^2\le\frac{3n}{N'(N'-1)}, \] and multiplying by $(N'/n)^2$ gives~\eqref{I:lo-rotation-bound}. The averaged one-replica law is a rotation-invariant probability measure on $S_{N'}$, hence uniform. By~\eqref{I:lo-density} its last $n$ coordinates have the density $f_{N,n}$, which converges pointwise to the standard Gaussian density; by Scheff\'e's lemma the laws converge to $\nu_n$ in total variation. \end{proof} \section{Proof of the lower bound}\label{I:lo:proof} The order of the limits matters, because the Gaussian integral over all of $\R^n$ in the numerator is not a bounded function of the overlaps. For fixed $n$ we first let $N\to\infty$ along a subsequence at a fixed cutoff $\Lambda$, then let the index $h$ of the approximating cascades tend to infinity, then let $\Lambda\to\infty$; the cavity dimension $n$ is sent to infinity last. \begin{proof}[Proof of \Cref{I:lo-lower}] We divide the proof into eight steps. \proofstep{1}{Increments.} Let $\Xi_N$ be the expected log partition function of the perturbed $N$-dimensional system with $M\sim\mathrm{Poi}(\alpha N)$ rows, averaged also over the perturbation parameters. By the Poissonization estimate of \Cref{I:lo:coords} and \Cref{I:lo-perturb}(i), \begin{equation}\label{I:lo:Xi} \frac{\Xi_N}N=F_N(U)+o(1). \end{equation} For every fixed $n$, \begin{equation}\label{I:lo-ass-inequality} \liminf_{N\to\infty}\frac{\Xi_N}N \ge\frac1n\liminf_{N\to\infty}(\Xi_{N+n}-\Xi_N). \end{equation} To verify this, let $\ell'$ be smaller than $n$ times the right side. Then $\Xi_{N+n}-\Xi_N\ge\ell'$ for all $N\ge N_0$. Every $N\ge N_0+n$ can be written as $N=N_1+jn$ with $N_0\le N_10$. By \Cref{I:lo-rotation}, the averaged one-replica law of the last $n$ coordinates of the $N'$-dimensional system converges to $\nu_n$, so the right side is $\E\min\{\norm{G'}^2/n,T\}$ for a standard Gaussian vector $G'$. For $h\ge h_0$, $\varepsilon^1\sim N(0,\varrho_hI_n)$ under $\cM^h$ with $\varrho_h\le\bar\varrho=P_0/c_0^2+1/c_0$ (\Cref{I:lo-positive-cont}), and $\E[(\norm{G'}^2/n)^2]=1+2/n\le3$. So removing the truncation changes the two sides by at most $3\bar\varrho^2/T$ and $3/T$. Letting $T\to\infty$ gives $\varrho_h\to1$. \item $F=\min\{(\varepsilon^1\cdot\varepsilon^2/n-R_{12})^2,T\}$. In the $N'$-dimensional system $\varepsilon^1\cdot\varepsilon^2/n$ is the quantity $R^{\rm cav}$ of \Cref{I:lo-rotation}, so $\E\langle F\rangle_{N'}\le3N'/(n(N'-1))$, and the left side of~\eqref{I:lo:transfer} is at most $3/n$. For $h\ge h_0$ the fourth moment of $\varepsilon^1\cdot\varepsilon^2/n-q^h_J$ under $\cM^h$ is bounded uniformly in $h$, since $0\le q^h_J\le1$ and $\varepsilon^1,\varepsilon^2$ have the law $N(0,\varrho_hI_n)$ with $\varrho_h\le\bar\varrho$. So removing the truncation costs $O(1/T)$ uniformly in $h$. This gives~\eqref{I:lo-full-constraints}. \end{enumerate} In particular, the row term in~\eqref{I:lo:unrestricted} is $\alpha u_\zeta(0,0;U)$. \proofstep{6}{The entropy.} For $h\ge h_0$, \Cref{I:gc:finite}(iv) gives \begin{equation}\label{I:lo:V-entropy} \cG(b_h,p^h)-\frac12\log b_h=\CS(\gamma_h)+\frac12\log\varrho_h, \qquad \gamma_h=\gamma_{m^h/\varrho_h}\in\cU, \end{equation} where $\gamma_h$ is the order parameter with atoms of mass $w^h_l$ at $m^h_l/\varrho_h$. The trial order parameters $\gamma_h$ are close to $\zeta$. The law $\zeta_h=\sum_lw^h_l\delta_{q^h_l}$ of $R_{12}$ under the $h$-th cascade array converges weakly to $\zeta$, hence in the Wasserstein distance $W_1$ on $[0,1]$. Under $\Prob_{\cM^h}$ the level $J$ has law $(w^h_l)$, so $(q^h_J,m^h_J/\varrho_h)$ is a coupling of $\zeta_h$ and $\gamma_h$, and since $0\le m^h_J\le m^h_{k_h}\le\varrho_h$, \[ W_1(\zeta_h,\gamma_h)\le\E\abs{q^h_J-m^h_J}+\abs{\varrho_h-1} \le\bigl(\E(q^h_J-m^h_J)^2\bigr)^{1/2}+\abs{\varrho_h-1}, \] with expectations under $\Prob_{\cM^h}$. By~\eqref{I:gc:overlap}, $m^h_J$ is the conditional mean of $\varepsilon^1\cdot\varepsilon^2/n$ given $J$, and $q^h_J$ is a function of $J$. So conditional Jensen's inequality and~\eqref{I:lo-full-constraints} give $\limsup_h\E(q^h_J-m^h_J)^2\le3/n$. Hence \begin{equation}\label{I:lo-l2} \limsup_{h\to\infty}W_1(\zeta,\gamma_h)\le\sqrt{3/n} . \end{equation} \proofstep{7}{The lower bound for fixed $n$.} Let $\delta>0$. By~\eqref{I:lo:unrestricted}, $\tilde\zeta=\zeta$, \eqref{I:lo:V-entropy}, $\varrho_h\to1$, and~\eqref{I:lo-l2}, there is $h$ such that $\gamma=\gamma_h\in\cU$ satisfies \[ \alpha u_\zeta(0,0;U)+\CS(\gamma)\le\frac{\ell_n}n+\delta,\qquad W_1(\zeta,\gamma)\le\sqrt{3/n}+\delta . \] \Cref{I:lo-phi-cont} gives a constant $C_U$ with $\abs{u_\zeta(0,0;U)-u_\gamma(0,0;U)}\le C_UW_1(\zeta,\gamma)$, so \[ \cP_U(\gamma)=\alpha u_\gamma(0,0;U)+\CS(\gamma) \le\frac{\ell_n}n+\alpha C_U\sqrt{3/n}+(1+\alpha C_U)\delta . \] Since $\gamma\in\cU$, the left side is at least $\cP_*(U)$. Letting $\delta\to0$ and using~\eqref{I:lo:Xi} and~\eqref{I:lo-ass-inequality}, \begin{equation}\label{I:lo:fixed-n} \liminf_{N\to\infty}F_N(U)\ge\frac{\ell_n}n \ge\cP_*(U)-\alpha C_U\sqrt{3/n}. \end{equation} \proofstep{8}{The limit $n\to\infty$.} The constant $C_U$ does not depend on $n$. Letting $n\to\infty$ in~\eqref{I:lo:fixed-n} proves \Cref{I:lo-lower}. The perturbation and the Poissonization were already removed in~\eqref{I:lo:Xi}. Neither a minimizer of $\cP_U$ nor the upper bound of \Cref{I:sec:upper} was used. \end{proof} \section{Proofs of the perturbation, continuity, and precision lemmas} \label{I:sec:lower-technical} We keep the notation of \Cref{I:lo:perturbations}. Throughout, $d\in\{N,N'\}$, $\cA$ is the set of rows of the system under consideration, $\bG_d$ is its matrix of rows, and constants may depend on $U$, $\alpha$, $n$, and $\Lambda$, but not on $N$. Since $M\in W_N$, the number of rows satisfies $\#\cA\le2\alpha N$, except in part~(i), where $M$ is Poisson, and in part~(v), where the $\pi$ added rows are included. All bounds on the log partition function hold uniformly for parameters $u_i,u_{v,m}$ in $[1/2,5/2]$, which allows difference quotients at the endpoints of $[1,2]$. We use the Gaussian Poincar\'e inequality \cite[Theorem~3.20]{BoucheronLugosiMassart2013}: for a standard Gaussian vector $\mathbf g$ in a Euclidean space and a continuously differentiable function $\Phi$ with $\E\norm{\nabla\Phi(\mathbf g)}^2<\infty$, $\Var\Phi(\mathbf g)\le\E\norm{\nabla\Phi(\mathbf g)}^2$. We first record an elementary estimate for replacing an activation by a smaller one on fresh rows. It is used in part~(v) of \Cref{I:lo-perturb} and in part~(iii) of \Cref{I:lo-cont}. \begin{lemma}\label{I:lo-trunc} Let $\langle\cdot\rangle$ be a random probability measure, and let $V_1\le V_2$ and $Y\ge0$ be random functions on its space such that $V_2\le A$ for a constant $A$, $V_1\ge-Y$, and $\E[\langle Y\rangle^2]<\infty$. Then, for every $T>0$, \[ 0\le\E\log\langle\e^{V_2}\rangle-\E\log\langle\e^{V_1}\rangle \le\e^T\,\E\langle\e^{V_2}-\e^{V_1}\rangle +\frac{\E[(A_++\langle Y\rangle)^2]}T . \] \end{lemma} \begin{proof} Put $Z_i=\langle\e^{V_i}\rangle$ and $\Delta=\log Z_2-\log Z_1\ge0$. By Jensen's inequality $\log Z_1\ge\langle V_1\rangle\ge-\langle Y\rangle$, and $\log Z_2\le A$; so both logarithms are integrable and $\Delta\le A_++\langle Y\rangle$. Since $\log x\le x-1$, also $\Delta\le(Z_2-Z_1)/Z_1$. On $\{Z_1\ge\e^{-T}\}$ this gives $\Delta\le\e^T(Z_2-Z_1)$. On $\{Z_1<\e^{-T}\}$ we have $\langle Y\rangle\ge-\log Z_1>T$, so $\Delta\le A_++\langle Y\rangle\le(A_++\langle Y\rangle)^2/T$. Taking expectations proves the claim. \end{proof} In both applications $V_1$ and $V_2$ are sums of activations over $\pi$ fresh rows $y_i$, and each $y_i(\rho)$ is standard Gaussian for one replica, so that $\E\langle(1+y_i^2)^2\rangle=6$. Recall the constant $C_0$ of~\eqref{I:lo:quadratic}. \begin{proof}[Proof of \Cref{I:lo-perturb}] We divide the proof into eight steps. \proofstep{1}{Tensor estimates.} We first record a bound that controls both the countable sum of Gaussian components and the derivatives of the Gaussian perturbation in the rows. If $T$ is a standard Gaussian multilinear form with $m$ slots, each of dimension at most $d+\#\cA$, its injective norm $\norm T_{\rm inj}$ (the supremum of $\abs{T(x_1,\dots,x_m)}$ over unit vectors) satisfies \begin{equation} \E\norm T_{\rm inj}^2\le Cm(d+\#\cA)\log(12m). \label{I:eq:lo-tensor} \end{equation} Indeed, a $1/(4m)$-net of the unit ball of $\R^D$ can be chosen with at most $(12m)^D$ points, so the product net over the $m$ slots has logarithmic cardinality at most $m(d+\#\cA)\log(12m)$. Changing one argument at a time from a point of the unit balls to a nearest net point changes $T$ by at most $\norm T_{\rm inj}/(4m)$ per slot, so $\norm T_{\rm inj}$ is at most $4/3$ times the maximum over the net. The values of $T$ at the net points are centered Gaussian with variance at most one, and $\E\max_{i\le K}\mathsf g_i^2\le4\log K+2$ for $K$ such variables $\mathsf g_i$, by $\E\e^{\mathsf g_i^2/4}\le\sqrt2$. This proves~\eqref{I:eq:lo-tensor}. Differentiating a tensor contraction in its response argument $y=\widehat\Psi_d(\rho)$ frees one slot and sums over at most $m$ positions; the other arguments are contracted with vectors of norm at most one, and the coefficients in~\eqref{I:lo:tensor-field} are at most one because $\sum_i\binom mi v_1^iv_2^{m-i}\le1$. Hence $\sup_\rho\norm{\partial_yh_{v,m}(\rho)}\le m\sum_{i\le m}\norm{T_{v,m,i}}_{\rm inj}$, and by~\eqref{I:eq:lo-tensor}, with no attempt to optimize the power of $m$, \begin{equation} \E\sup_\rho\norm{\partial_yh_{v,m}(\rho)}^2 \le C(m+1)^5\log(12(m+1))(d+\#\cA). \label{I:eq:lo-feature-derivative} \end{equation} These polynomial factors are summable against $c_{v,m}=2^{-j(v)-m}$. In the same way $\sup_\rho\abs{h_{v,m}(\rho)}\le\sum_{i\le m}\norm{T_{v,m,i}}_{\rm inj}$, so $\bigl(\E\sup_\rho\abs{h_{v,m}(\rho)}^2\bigr)^{1/2}\le C(m+1)^{3/2}\log(12(m+1))^{1/2}(d+\#\cA)^{1/2}$ by the Cauchy--Schwarz inequality and~\eqref{I:eq:lo-tensor}. Hence $\sum_{v,m}c_{v,m}\bigl(\E\sup_\rho\abs{h_{v,m}}^2\bigr)^{1/2}<\infty$. By Tonelli's theorem the series defining the Gaussian part converges absolutely almost surely, uniformly in $\rho$, and by Minkowski's inequality the supremum over $\rho$ of its tail after finitely many components tends to zero in $L^2$. The same summable majorants, with the polynomial factors of~\eqref{I:eq:lo-feature-derivative}, apply to the derivatives of the finite truncations used below. Every statement below is first proved with finitely many components and then passes to the limit. \proofstep{2}{Part (i).} Since $\abs{\tau_i(x)}\le C_\tau(1+x^2)$, the scalar part changes $\log Z$ by at most $C\xi_d\sum_a(1+x_a(\rho)^2)\le C\xi_d(\#\cA+\norm{\bG_d}_{\rm op}^2)$, whose expectation is at most $C\xi_dN$ by \Cref{I:lo:opnorm} (averaged over $M$). For the Gaussian part, Jensen's inequality, conditionally on everything except the tensors, shows that its expected effect lies between $0$ and $s_d^2\sup_\rho\sum_{v,m}c_{v,m}^2u_{v,m}^2Q_{v,m}(\rho,\rho)/2\le Cs_d^2$: the lower bound because the Gibbs average of the Gaussian part under the unperturbed measure has mean zero, the upper bound because $\E\e^{h}=\e^{\Var(h)/2}$ for a centered Gaussian variable $h$. The total expected cost is therefore $O(d^{3/4}+d^{2/3})=o(d)$. \proofstep{3}{Concentration of the free energy.} For the reference and the $N'$-dimensional system with $\cA=\{1,\dots,M\}$, we show that \begin{equation} \Var(\log Z)\le Cd,\qquad v_d:=\sup\E\abs{\log Z-\E\log Z}\le C\sqrt d, \label{I:eq:lo-variance} \end{equation} where the supremum is over the parameters. Apply the Gaussian Poincar\'e inequality, with finitely many tensors, in the rows and the tensor entries. Its right side is the expected squared gradient, which has three parts. The gradient in the rows through the activation contributes at most $C(M+\norm{\bG_d}_{\rm op}^2)$, because $\sum_i\rho_i^2=d$ and the first derivative of $U^\xi$ grows at most linearly. The gradient in the tensor entries contributes at most $Cs_d^2$, because each $h_{v,m}$ is linear in its tensor with coefficient vector of norm at most one. For the gradient in the rows through the response vector, write $y=\widehat\Psi_d(\rho)$; then \[ \partial_{g_{ai}}h_{v,m} =\bigl[D\varpi(\Psi_d)^{\mathsf T}\partial_yh_{v,m}\bigr]_a\, U''(x_a(\rho))\,\frac{\rho_i}{d\sqrt{C_X}} . \] Summing the squares over $a$ and $i$ and using $\norm\rho^2=d$, \eqref{I:eq:lo-feature-derivative} bounds the expected contribution by $Cs_d^2(1+M/d)$. For the reference system, the tilt $nB_N/2$ adds a row gradient of squared norm at most $C_nM/d^2$. Since $\E\norm{\bG_d}_{\rm op}^2\le C(M+d)$ by \Cref{I:lo:opnorm} and $M\le2\alpha N$, the variance is at most $C(M+d)+Cs_d^2(1+M/d)\le Cd$. The bound passes to infinitely many components by Step~1, which proves~\eqref{I:eq:lo-variance}. \proofstep{4}{Part (ii).} Steps~4 and~5 follow the proof of the Ghirlanda--Guerra identities in \cite[Theorem~3.2]{PanchenkoBook}; see also \cite[Theorem~2]{Panchenko2015}. Consider a scalar parameter $u=u_i$. In a $d$-dimensional system it enters the Hamiltonian as $d\xi_d\,u\,A(\rho)$ with $A=d^{-1}\sum_a\tau_i(x_a(\rho))$; for the reference system $d=N'$, $x_a=S_a$, and $A=A_i$. The function $u\mapsto\log Z$ is convex, with first derivative $d\xi_d\langle A\rangle$ and second derivative $(d\xi_d)^2\langle(A-\langle A\rangle)^2\rangle$. Integrating the second derivative over $[1,2]$ and using that $\E\langle\abs A\rangle$ is bounded by~\eqref{I:lo:observable-bounds}, \[ \int_1^2\E\langle(A-\langle A\rangle)^2\rangle\dd u\le\frac C{d\xi_d}. \] For the fluctuation of $\langle A\rangle$ with the disorder we use the following consequence of convexity: if $f,g$ are convex and differentiable, then for $t>0$ \[ \abs{f'(u)-g'(u)}\le g'(u+t)-g'(u-t) +\frac{\abs{f-g}(u+t)+\abs{f-g}(u-t)+2\abs{f-g}(u)}t . \] (Bound $f'(u)$ above by $(f(u+t)-f(u))/t$ and below by $(f(u)-f(u-t))/t$, compare with $g$, and use that $g'$ is nondecreasing.) Apply it to $f=\log Z$ and $g=\E\log Z$ as functions of $u$ and integrate over $u\in[1,2]$. For $00$ with $\sum_i\mathsf w_i\bar c_i<\infty$, and let $\epsilon_N(M)$ be the average over the parameters of $\sum_i\mathsf w_i\mathsf d_i$. By dominated convergence, applied to the weighted sum of the suprema over $M\in W_N$ of the averaged defects, $\bar\epsilon_N=\sup_{M\in W_N}\epsilon_N(M)\to0$. By Markov's inequality, the set $\mathcal G_N(M)$ where $\sum_i\mathsf w_i\mathsf d_i\le\bar\epsilon_N^{1/2}$ has probability at least $1-\bar\epsilon_N^{1/2}$, and on $\mathcal G_N(M)$ each fixed defect $\mathsf d_i$ is at most $\bar\epsilon_N^{1/2}/\mathsf w_i$. None of these defects involves $\Lambda$, so neither does $\mathcal G_N(M)$. This proves~(iii). \proofstep{6}{Part (iv)(a).} We first retain a finite set $\mathcal B$ of components $\beta=(v,m)$; no constant below depends on $\mathcal B$. Write $s=s_{N'}$, $\lambda_\beta=c_{v,m}u_{v,m}$, and $h_\beta=h_{v,m}$, and let $C_\varpi$ be a common bound for the operator norm of $D\varpi(z)$ and the norm of the bilinear map $D^2\varpi(z)$, uniform in $z$ and in the dimension. Use the path $\rho^t=\rho(\sigma,t\varepsilon)$, $0\le t\le1$, on $S_{N'}$. Keep the activation part of the Hamiltonian at $\rho(\sigma,\varepsilon)$ fixed, and evaluate the Gaussian part $P_t(\sigma,\varepsilon)=s\sum_\beta\lambda_\beta h_\beta(\rho^t)$ at $\rho^t$. Let $Z_t$ be the partition function restricted to $\varepsilon\in\mathbb B_\Lambda$ and $\langle\cdot\rangle_t$ its Gibbs measure, and assume $N\ge2\Lambda n$. For a replica $(\sigma^p,\varepsilon^p)$ put $\rho_p=\rho(\sigma^p,t\varepsilon^p)$, $r_p=\rho_p/\sqrt{N'}$, $\zeta^p=\varepsilon^p/\sqrt{N'}$, and \[ x_{pa}=x_a(\rho_p),\quad \bar x_{pa}=x_a(\rho(\sigma^p,\varepsilon^p)),\quad \Psi_p=\Psi_{N'}(\rho_p),\quad y_p=\varpi(\Psi_p),\quad J_p=D\varpi(\Psi_p). \] The fields $\bar x_{pa}$ enter the activation part, and $\sum_j\abs{\zeta^p_j\zeta^q_j}\le\Lambda n/N'$. A dot denotes $\partial_t$. \proofpart{The derivative.} By~\eqref{I:lo:tensor-field}, $F_\beta(\rho)\cdot F_\beta(\rho')$ is the polynomial $(v_1r\cdot r'+v_2y\cdot y')^m$ in the spin arguments $r=\rho/\sqrt{N'}$, $r'$ and the response arguments $y,y'$. Differentiating in the first pair of arguments, \begin{equation}\label{I:lo:pq-term} D^\beta_{pq}:=\partial_tF_\beta(\rho_p)\cdot F_\beta(\rho_q) =mK_{pq}^{m-1}\bigl(v_1\dot r_p\cdot r_q+v_2\dot y_p\cdot y_q\bigr),\qquad K_{pq}=v_1r_p\cdot r_q+v_2y_p\cdot y_q, \end{equation} and $\abs{K_{pq}}\le1$. Since $\partial_t\log Z_t=\langle\partial_tP_t\rangle_t$, Gaussian integration by parts in the tensor entries, conditionally on the rows, gives \[ \frac{\dd}{\dd t}\E\log Z_t =s^2\sum_\beta\lambda_\beta^2\,\E\langle D^\beta_{11}-D^\beta_{12}\rangle_t . \] Let $f$ be a bounded measurable function of the configurations of $r$ replicas. The derivative of $\E\langle f\rangle_t$ is $\E\langle f\sum_{p\le r+1}a_p\partial_tP_t(\sigma^p,\varepsilon^p)\rangle_t$, with $a_p=1$ for $p\le r$ and $a_{r+1}=-r$. Integration by parts in the tensor entries, now with $r+1$ replicas in the Gibbs average, gives \[ \frac{\dd}{\dd t}\E\langle f\rangle_t =s^2\sum_\beta\lambda_\beta^2\sum_{p\le r+1}\sum_{q\le r+2} a_pb_q\,\E\langle fD^\beta_{pq}\rangle_t , \] with $b_q=1$ for $q\le r+1$ and $b_{r+2}=-(r+1)$, so that $\sum_p\abs{a_p}=2r$ and $\sum_q\abs{b_q}=2(r+1)$. With finitely many components, $\partial_tP_t$ and all derivatives in the rows used below are bounded by polynomials in $\norm{[\bG,\bG']}_{\rm op}$, where $[\bG,\bG']$ is the $M\times N'$ matrix with rows $(g_a,g'_a)$, and the injective norms of the tensors, which have moments of all orders. This justifies the differentiation under $\E$ and the integrations by parts. Both derivatives are therefore controlled by the following bound: if $f$ is a bounded measurable function of the configurations of $k$ replicas, which depends neither on $t$ nor on the disorder, and $p,q\le k$, then \begin{equation}\label{I:lo:pair-bound} \abs{\E\langle fD^\beta_{pq}\rangle_t}\le C_{n,\Lambda}k\norm f_\infty \Bigl(\frac{m^2}N+\frac{ms}{N^{3/2}}\Bigr). \end{equation} \proofpart{Spin and radial parts.} We have $\dot\rho_p=(\dot\chi_p\sigma^p,\varepsilon^p)$ with $\chi_p=\chi(t\varepsilon^p)$ and $\dot\chi_p=-t\norm{\varepsilon^p}^2/(N\chi_p)$. Since $N\ge2\Lambda n$, we have $1/2\le\chi_p^2\le N'/N\le2$, so $\abs{\dot\chi_p}\le2\Lambda n/N$ and $\chi_p\le\sqrt2$. Since $\norm{\rho_p}^2=N'$, the diagonal term $\dot r_p\cdot r_p$ vanishes, and for $p\ne q$ \[ \abs{\dot r_p\cdot r_q} =\frac{\abs{\dot\chi_p\chi_q\,\sigma^p\cdot\sigma^q +t\,\varepsilon^p\cdot\varepsilon^q}}{N'}\le\frac{4\Lambda n}N . \] So the spin part of $D^\beta_{pq}$ is at most $4\Lambda nm/N$. For the response part, $\dot y_p=J_p\dot\Psi_p$ and $\dot\Psi_{pa}=U''(x_{pa})\dot x_{pa}/\sqrt{C_XN'}$, where by~\eqref{I:lo:D-fields} \[ \dot x_{pa}=\dot\chi_pS_a(\sigma^p)+\sum_{j\le n}g'_{aj}\zeta^p_j . \] Put $Y^{pq}=J_p^{\mathsf T}y_q\in\R^M$, so that $\norm{Y^{pq}}\le C_\varpi$ and $\dot y_p\cdot y_q=\sum_aY^{pq}_aU''(x_{pa})\dot x_{pa}/\sqrt{C_XN'}$. Since $\sum_aS_a(\sigma)^2\le\norm\bG_{\rm op}^2$, the radial part of $mv_2K_{pq}^{m-1}\dot y_p\cdot y_q$ is at most $2\Lambda nmLC_\varpi\norm\bG_{\rm op}/(N\sqrt{C_XN'})$, and $\E\norm\bG_{\rm op}\le C\sqrt N$ by \Cref{I:lo:opnorm}. Both parts contribute at most $C_{n,\Lambda}m\norm f_\infty/N$ to~\eqref{I:lo:pair-bound}. \proofpart{Integration by parts in the fresh columns.} The remaining part of $D^\beta_{pq}$ is $\sum_{a,j}g'_{aj}A^{pq}_{aj}$, where \[ A^{pq}_{aj}=\frac{mv_2}{\sqrt{C_XN'}}\,\zeta^p_j\,K_{pq}^{m-1}\, U''(x_{pa})\,Y^{pq}_a . \] Write $\partial_{aj}=\partial/\partial g'_{aj}$ and let $\mathcal H_t(\sigma,\varepsilon)=\sum_aU^\xi(x_a(\rho(\sigma,\varepsilon))) +P_t(\sigma,\varepsilon)$ be the Hamiltonian. The function $f$ does not depend on $g'_{aj}$. Gaussian integration by parts in $g'_{aj}$ gives \begin{equation}\label{I:lo:fresh-ibp} \E\Bigl\langle f\sum_{a,j}g'_{aj}A^{pq}_{aj}\Bigr\rangle_t =\sum_{a,j}\E\langle f\,\partial_{aj}A^{pq}_{aj}\rangle_t +\sum_{\ell\le k+1}\eta_\ell\sum_{a,j} \E\bigl\langle fA^{pq}_{aj}\,\partial_{aj}\mathcal H_t(\sigma^\ell,\varepsilon^\ell) \bigr\rangle_t , \end{equation} where $\eta_\ell=1$ for $\ell\le k$ and $\eta_{k+1}=-k$. The terms $\ell\le k$ come from the Gibbs densities of the replicas of $fA^{pq}$ (the same replica when $\ell\in\{p,q\}$, another replica otherwise), and the term $\ell=k+1$ comes from the normalization; $\sum_\ell\abs{\eta_\ell}=2k$. Only the $a$-th field depends on $g'_{aj}$, and \[ \partial_{aj}x_{pa}=t\zeta^p_j,\qquad \partial_{aj}\bar x_{pa}=\zeta^p_j,\qquad \partial_{aj}\Psi_p=\frac{t\zeta^p_jU''(x_{pa})}{\sqrt{C_XN'}}\,e_a, \] where $e_a$ is the $a$-th unit vector of $\R^M$. In the following bounds we use $\abs{U''},\abs{U'''}\le L$, $\abs{K_{pq}}\le1$, $t,v_2\le1$, and $\sum_j\abs{\zeta^p_j\zeta^q_j}\le\Lambda n/N'$. \proofpart{Derivatives of the coefficient.} The coefficient $A^{pq}_{aj}$ depends on $g'_{aj}$ through four factors. \begin{enumerate}[label=\textup{(\alph*)}] \item Through $U''(x_{pa})$ the derivative is $U'''(x_{pa})t\zeta^p_j$. This is the only term with the $\ell^1$ norm of $Y^{pq}$, and $\sum_a\abs{Y^{pq}_a}\le\sqrt M\norm{Y^{pq}}$. Its sum over $a,j$ is at most $mL\Lambda nC_\varpi\sqrt M/(\sqrt{C_X}N'^{3/2})$, which is $O_{n,\Lambda}(m/N)$ since $M\le2\alpha N$. \item Through the cutoff Jacobian $J_p$ in $Y^{pq}_a=\sum_b(J_p)_{ba}y_{qb}$, the derivative of $Y^{pq}_a$ is $t\zeta^p_jU''(x_{pa})\,D^2\varpi(\Psi_p)[e_a,e_a]\cdot y_q/\sqrt{C_XN'}$. Each term $\abs{D^2\varpi(\Psi_p)[e_a,e_a]\cdot y_q}$ is at most $C_\varpi$, so the sum over $a$ of this Hessian trace costs a factor $M$. The sum over $a,j$ is at most $mL^2\Lambda nMC_\varpi/(C_XN'^2)=O_{n,\Lambda}(m/N)$. \item Through the other response vector $y_q$, whose derivative is $t\zeta^q_jU''(x_{qa})J_qe_a/\sqrt{C_XN'}$, the derivative of $Y^{pq}_a$ is $t\zeta^q_jU''(x_{qa})(J_p^{\mathsf T}J_q)_{aa}/\sqrt{C_XN'}$. Since $\sum_a\abs{(J_p^{\mathsf T}J_q)_{aa}}\le\norm{J_p}_{\rm F}\norm{J_q}_{\rm F} \le MC_\varpi^2$, the sum over $a,j$ is at most $mL^2\Lambda nMC_\varpi^2/(C_XN'^2)=O_{n,\Lambda}(m/N)$. When $p=q$, (b) and~(c) are the two derivatives of $J_p^{\mathsf T}y_p$. \item Through the remaining tensor slots, that is, through $K_{pq}^{m-1}$, with \[ \partial_{aj}K_{pq}=\frac{v_2t}{\sqrt{C_XN'}} \bigl(\zeta^p_jU''(x_{pa})Y^{pq}_a+\zeta^q_jU''(x_{qa})Y^{qp}_a\bigr), \] since the spin part of $K_{pq}$ does not depend on the rows. The two row vectors in each product are paired before norms are taken: $\sum_a\abs{Y^{pq}_a}(\abs{Y^{pq}_a}+\abs{Y^{qp}_a})\le2C_\varpi^2$, with no factor $M$. The sum over $a,j$ is at most $2m(m-1)L^2\Lambda nC_\varpi^2/(C_XN'^2)=O_{n,\Lambda}(m^2/N^2)$. \end{enumerate} The first sum on the right of~\eqref{I:lo:fresh-ibp} is therefore at most $C_{n,\Lambda}\norm f_\infty m^2/N$. \proofpart{The activation part of the Gibbs density.} The activation part of $\partial_{aj}\mathcal H_t(\sigma^\ell,\varepsilon^\ell)$ is $(U^\xi)'(\bar x_{\ell a})\zeta^\ell_j$, and \[ \sum_{a,j}A^{pq}_{aj}(U^\xi)'(\bar x_{\ell a})\zeta^\ell_j =\frac{mv_2K_{pq}^{m-1}\,\zeta^p\cdot\zeta^\ell}{\sqrt{C_XN'}} \sum_aU''(x_{pa})Y^{pq}_a(U^\xi)'(\bar x_{\ell a}). \] By~\eqref{I:lo:class} and the growth of the derivatives of $\tau_1$ and $\tau_2$, $\abs{(U^\xi)'(x)}\le C(1+\abs x)$. Since $\sum_a\bar x_{\ell a}^2\le\norm{[\bG,\bG']}_{\rm op}^2$ for every configuration, the Cauchy--Schwarz inequality bounds this sum by $CmL\Lambda nC_\varpi(\sqrt M+\norm{[\bG,\bG']}_{\rm op})/(\sqrt{C_X}N'^{3/2})$. By \Cref{I:lo:opnorm}, $\E\norm{[\bG,\bG']}_{\rm op}\le C\sqrt N$, so each replica $\ell$ contributes $C_{n,\Lambda}\norm f_\infty m/N$. \proofpart{The Gaussian part of the Gibbs density.} Let $\Gamma_\ell=\sum_{\beta'}\lambda_{\beta'}\partial_yh_{\beta'}$, evaluated at $\rho_\ell$, be the response gradient of the Gaussian part. Then $\partial_{aj}P_t(\sigma^\ell,\varepsilon^\ell)=s\Gamma_\ell\cdot\partial_{aj}y_\ell =st\zeta^\ell_jU''(x_{\ell a})(J_\ell^{\mathsf T}\Gamma_\ell)_a/\sqrt{C_XN'}$, and \[ \sum_{a,j}A^{pq}_{aj}\,\partial_{aj}P_t(\sigma^\ell,\varepsilon^\ell) =\frac{stmv_2K_{pq}^{m-1}\,\zeta^p\cdot\zeta^\ell}{C_XN'} \sum_aU''(x_{pa})U''(x_{\ell a})Y^{pq}_a(J_\ell^{\mathsf T}\Gamma_\ell)_a . \] We bound $\Gamma_\ell$ by a pathwise envelope. By Step~1, $\norm{\partial_yh_{v,m}}\le m\sum_{i\le m}\norm{T_{v,m,i}}_{\rm inj}$ at all arguments in the unit balls. Hence $\norm{\Gamma_\ell}\le\mathsf W$ for every configuration, where $\mathsf W=\sum_{v,m}\lambda_{v,m}m\sum_{i\le m}\norm{T_{v,m,i}}_{\rm inj}$ does not depend on the Gibbs measure. By~\eqref{I:eq:lo-tensor} with $d=N'$ and $\#\cA=M$, \[ \E\mathsf W\le B_*\sqrt{N'+M},\qquad B_*=\frac52C^{1/2}\sum_{v,m}c_{v,m}\,m(m+1)\bigl(m\log(12m)\bigr)^{1/2}<\infty . \] The Cauchy--Schwarz inequality in $a$ bounds the last sum by $L^2C_\varpi^2\mathsf W$, so each replica $\ell$ contributes at most $smL^2\Lambda nC_\varpi^2\norm f_\infty\E\mathsf W/(C_XN'^2) \le C_{n,\Lambda}\norm f_\infty ms/N^{3/2}$. Bounding the coordinates of $J_\ell^{\mathsf T}\Gamma_\ell$ one at a time would lose a factor $\sqrt M$. Summing over $\ell$ with the weights $\eta_\ell$ and adding the previous parts proves~\eqref{I:lo:pair-bound}. \proofpart{Summation over the components.} Since $u_{v,m}\le5/2$ and $c_{v,m}=2^{-j(v)-m}$ with $j(v)\ge1$, \[ \sum_{v,m}\lambda_{v,m}^2m^2\le\frac{25}{4}\sum_{j\ge1}4^{-j}\sum_{m\ge1}m^24^{-m} =\frac{125}{81} . \] By~\eqref{I:lo:pair-bound} with $k=2$ and $m\le m^2$, the derivative of $\E\log Z_t$ is at most $C_{n,\Lambda}(s^2/N+s^3/N^{3/2})$. Integrating over $t\in[0,1]$, the complete error is bounded by \begin{equation} C_{n,\Lambda}\bigl(s_{N'}^2/N+s_{N'}^3/N^{3/2}\bigr)=o(1). \label{I:eq:lo-cavity-perturbation} \end{equation} For a bounded measurable function $f$ of the spin overlaps and cavity coordinates of $r$ replicas, \eqref{I:lo:pair-bound} applies with $k=r+2$, and the integration by parts in the fresh columns introduces the replica $r+3$. So $\abs{\partial_t\E\langle f\rangle_t}\le4r(r+1)(r+2)C_{n,\Lambda}\norm f_\infty (s^2/N+s^3/N^{3/2})$. \proofpart{Infinitely many components and rare events.} The bounds above do not depend on $\mathcal B$. Let $\mathcal B_K$ be the set of the first $K$ components and $\delta_K=s\sum_{\beta\notin\mathcal B_K}\lambda_\beta\sup_\rho\abs{h_\beta(\rho)}$. By Step~1, $\delta_K\to0$ almost surely and in $L^1$. Truncating the Gaussian part to $\mathcal B_K$ changes $\log Z_t$ by at most $\delta_K$, and it changes $\langle f\rangle_t$ by at most $\norm f_\infty\min(2,\e^{2r\delta_K}-1)$, since the Gibbs density of $r$ replicas changes by a factor in $[\e^{-2r\delta_K},\e^{2r\delta_K}]$. By dominated convergence, the bounds pass to the full series at $t=0$ and $t=1$. No restriction to $\Omega_N$ was used: the bounds on $\varpi$ and its derivatives hold everywhere, and $\norm\bG_{\rm op}$, $\norm{[\bG,\bG']}_{\rm op}$, and $\mathsf W$ enter only through their unconditional expectations. \proofstep{7}{Part (iv)(b).} In the representation through the fields $x_a=S_a+\eta_a$, the $N$-dimensional system has the activation $U^{\xi_N}(x_a)$ and the Gaussian part $s_N\sum_\beta\lambda_\beta h_\beta$ with the response vector $\varpi(\Psi^1)$, where \[ \Psi^t(\sigma)=\frac{(U'(S_a(\sigma)+t\eta_a(\sigma)))_{a\le M}}{\sqrt{C_XN}} \qquad(0\le t\le1). \] Conditionally on the rows, each Gaussian part considered below consists of independent centered Gaussian processes on $S_N$ with covariances $(v_1R_N(\sigma,\sigma')+v_2y(\sigma)\cdot y(\sigma'))^m$ for some response vector $y$ with values in the unit ball. The expected log partition function depends on the Gaussian part only through these covariances. We retain finitely many components and pass to the limit as in Step~6, and we make four replacements. \proofpart{The response vector.} Replace $\varpi(\Psi^1)$ by $\varpi(\Psi^t)$, keeping the activation at $S_a+\eta_a$, and let $t$ run from $1$ to $0$. Define $y_p$, $J_p$, $K_{pq}$, and $Y^{pq}$ as in Step~6, with $r_p=\sigma^p/\sqrt N$, $\Psi_p=\Psi^t(\sigma^p)$, $x_{pa}=S_a(\sigma^p)+t\eta_a(\sigma^p)$, and $\zeta^p=c_N\sigma^p\in\R^N$. Then $\sum_i\abs{\zeta^p_i\zeta^q_i}\le c_N^2N=n/N'$. The spin argument $\sigma/\sqrt N$ does not move, so~\eqref{I:lo:pq-term} has no spin or radial part, and $\dot x_{pa}=\eta_a(\sigma^p)=\sum_ig''_{ai}\zeta^p_i$. Hence $D^\beta_{pq}=\sum_{a,i}g''_{ai}A^{pq}_{ai}$ with \[ A^{pq}_{ai}=\frac{mv_2}{\sqrt{C_XN}}\,\zeta^p_i\,K_{pq}^{m-1}\, U''(x_{pa})\,Y^{pq}_a , \] the coefficient of Step~6 with $\varepsilon^p/\sqrt{N'}$ replaced by $c_N\sigma^p$ and $N'$ replaced by $N$ in the normalization of the response vector. Integrate by parts in the entries $g''_{ai}$ of $\bG''$. Identity~\eqref{I:lo:fresh-ibp} holds with $\partial_{g''_{ai}}x_{pa}=t\zeta^p_i$, and the derivative of the activation field $\bar x_{\ell a}=S_a(\sigma^\ell)+\eta_a(\sigma^\ell)$ is $\zeta^\ell_i$. The bounds (a)--(d) on the coefficient and the bounds on the activation and Gaussian parts of the Gibbs density hold as in Step~6, with three changes: $\Lambda n/N'$ becomes $n/N'$; $\norm{[\bG,\bG']}_{\rm op}$ becomes $\norm\bG_{\rm op}+c_N\sqrt N\norm{\bG''}_{\rm op}$, which bounds $\norm{(\bar x_{\ell a})_a}$ and has expectation at most $C\sqrt N$; and $\mathsf W$ is formed with the tensors of dimension $N+M$, so $\E\mathsf W\le B_*\sqrt{N+M}$. Thus~\eqref{I:lo:pair-bound} holds with $C_n$ and $s=s_N$, and the summation of Step~6 bounds the change of $\E\log Z$ by $C_n(s_N^2/N+s_N^3/N^{3/2})$. \proofpart{Normalization and scaling.} At $t=0$ the response vector is $y^0=\varpi(\Psi^0)$, while that of the reference perturbation is $y^{\rm ref}=\varpi(\Psi^{\rm ref})$ with $\Psi^{\rm ref}(\sigma)=(U'(\sqrt{N'/N}\,S_a(\sigma)))_a/\sqrt{C_XN'}$ (see \Cref{I:lo:perturbations}). The two differ in the normalization ($N$ instead of $N'$) and in the factor $\sqrt{N'/N}$ in the argument of $U'$. Since $1/\sqrt N-1/\sqrt{N'}\le n/(2N^{3/2})$, $\abs{U'(\sqrt{N'/N}\,S)-U'(S)}\le L\abs Sn/(2N)$, and $\norm{(U'(S_a))_a}\le L(\sqrt M+\norm\bG_{\rm op})$, and since $\varpi$ is $C_\varpi$-Lipschitz, \[ \sup_\sigma\norm{y^0(\sigma)-y^{\rm ref}(\sigma)}\le\delta_N,\qquad \delta_N=\frac{C_\varpi Ln(\sqrt M+\norm\bG_{\rm op})}{\sqrt{C_X}N^{3/2}}, \qquad \E\delta_N\le\frac{Cn}N . \] All response vectors lie in the unit ball, and $\abs{k^m-k'^m}\le m\abs{k-k'}$ for $k,k'\in[-1,1]$, so the two covariances of each component differ by at most $2m\delta_N$. Let $h^0_\beta$ and $h^{\rm ref}_\beta$ be conditionally independent Gaussian processes with these covariances, and interpolate with $\sqrt{1-t}\,h^0_\beta+\sqrt t\,h^{\rm ref}_\beta$. Gaussian integration by parts gives the derivative $(s_N^2/2)\sum_\beta\lambda_\beta^2\E\langle\Delta Q^\beta_{11}-\Delta Q^\beta_{12}\rangle$, where $\Delta Q^\beta$ is the difference of the two covariances. Its absolute value is at most $2s_N^2\E\delta_N\sum_\beta\lambda_\beta^2m=O_n(s_N^2/N)$. \proofpart{The strengths.} The derivative of the expected log partition function in $s$ is $\sum_\beta\lambda_\beta\E\langle h_\beta\rangle =s\sum_\beta\lambda_\beta^2\E\langle Q^\beta_{11}-Q^\beta_{12}\rangle$, by integration by parts, hence $O(s)$. Since $\abs{s_{N'}-s_N}\le nN^{-2/3}$, changing $s_N$ to $s_{N'}$ costs $O(nN^{-1/3})$. The derivative in $\xi$ is $\E\langle\sum_a(u_1\tau_1+u_2\tau_2)(x_a)\rangle$, which is at most $C(M+\E\norm\bG_{\rm op}^2+c_N^2N\E\norm{\bG''}_{\rm op}^2)=O(N)$. Since $\abs{\xi_{N'}-\xi_N}\le nN^{-5/4}$, changing $\xi_N$ to $\xi_{N'}$ costs $O(nN^{-1/4})$. As in Step~6, no restriction to $\Omega_N$ is needed. \proofstep{8}{Part (v).} When rows are added, the response vector gains coordinates, and we realize the tensors of the system with $M$ rows as the restrictions of those of the system with $M+\pi$ rows to the old response coordinates. On the event $\Omega'_N$ of \Cref{I:lo:perturbations}, which depends only on the rows and on $\pi$, the map $\varpi$ is the identity for the old and the new response vectors, by the choice of $C_X$. There the Gaussian part of the perturbation changes by a centered Gaussian field that is linear in the tensor entries involving a new coordinate, and these entries are independent of the old ones. Since the logarithm of a Gibbs average of $\e^{h}$ is at least the Gibbs average of $h$, averaging over these entries shows that this change has a nonnegative effect on the expected log partition function. On the complement, the change of the log partition function is at most $s_{N'}\sum_{v,m}c_{v,m}u_{v,m}$ times the suprema of the old and new components in absolute value, whose second moment is at most $CN^2$ by Step~1, while $\Prob(\Omega_N'^{\,c})\le C_n\e^{-N/2}$; by the Cauchy--Schwarz inequality the complement contributes $o(1)$. The added rows carry the activation $U^{\xi_{N'}}$. Since $\abs{\tau_i(x)}\le C_\tau(1+x^2)$, $u_1\le5/2$, $u_2\ge0$, and $\tau_2\ge0$, we have $U^{\xi_{N'}}\ge U-c(1+x^2)$ with $c=4C_\tau\xi_{N'}$, and $c\le1$ for $N$ large; the expected logarithmic increment is monotone in the activation. The fields $y_i$ of the added rows are standard Gaussian for each configuration and independent of $\langle\cdot\rangle_{N'}$. Conditionally on $\pi$, apply \Cref{I:lo-trunc} to $\langle\cdot\rangle_{N'}$ with $V_2=\sum_{i\le\pi}U(y_i)$, $V_1=\sum_{i\le\pi}[U(y_i)-c(1+y_i^2)]$, $A=\pi U_{\max}$, and $Y=(C_0+1)\sum_{i\le\pi}(1+y_i^2)$. Then $\E[\langle Y\rangle^2]\le\E\langle Y^2\rangle\le6(C_0+1)^2\pi^2$, and $\e^{V_2}-\e^{V_1}\le\e^{\pi U_{\max}^+}c\sum_{i\le\pi}(1+y_i^2)$ has expectation at most $2\pi c\,\e^{\pi U_{\max}^+}$. Averaging over $\pi\sim\mathrm{Poi}(\alpha n)$, whose exponential moments are finite, we obtain for every $T>0$ \[ 0\le\E\log\Big\langle\e^{\sum_{i\le\pi}U(y_i)}\Big\rangle_{N'} -\E\log\Big\langle\e^{\sum_{i\le\pi}[U(y_i)-c(1+y_i^2)]}\Big\rangle_{N'} \le C_n\bigl(c\,\e^T+T^{-1}\bigr), \] uniformly in the Gibbs measure. Choosing $T=T_N\to\infty$ so slowly that $\xi_{N'}\e^{T_N}\to0$, we see that replacing $U^{\xi_{N'}}$ by $U$ on the added rows costs $o(1)$. This proves the first assertion of~(v). For the uniform lower bound, take $\Lambda=2$ in the coordinate term. After~\eqref{I:eq:lo-cavity-perturbation}, \Cref{I:lo-perturb}(iv)(b) and the interpolation of \Cref{I:lo:cavity-terms}, whose errors are uniform in the parameters and in $M\in W_N$, the coordinate term is at least the difference~\eqref{I:lo:coordinate-N} minus $o(1)$. Jensen's inequality, for the Gibbs average and for the normalized restriction of $\nu_n$ to the centered ball $\mathbb B_2$, bounds the logarithm of the numerator below by $\log\nu_n(\mathbb B_2)-C_n\langle1+\abs{b_N}\rangle$, whose expectation is at least $-C_n$ by~\eqref{I:lo:observable-bounds}. Jensen's inequality in the field $y$ bounds the expected logarithm of the denominator above by $\E\log\langle\e^{nX_N(\sigma,\sigma)/2}\rangle\le n\E\sup_\sigma X_N(\sigma,\sigma)/2\le C_n$. By the first assertion of~(v) and Jensen's inequality for the fresh rows, the row term is at least $\alpha n\,\E U(G)-o(1)\ge-C_n$. Combining these bounds proves the lower bound in the last assertion of~(v) for $N\ge N_0(n)$. For the upper bound on the absolute value, for any number of rows $\abs{\E\log Z}\le C(d+\#\cA)+Cs_d^2$ in each system: without its Gaussian part, the Hamiltonian is at most $C(\#\cA+\norm{\bG_d}_{\rm op}^2)+C_n\#\cA/d$ in absolute value, and Jensen's inequality in the tensors bounds the effect of the Gaussian part between $0$ and $Cs_d^2$. \end{proof} \begin{proof}[Proof of \Cref{I:lo-precision}] We fix the perturbation parameters and the number $M\le2\alpha N$ of rows. The Gibbs measure $\langle\cdot\rangle_{\rm ref}$ has density proportional to $\e^{H}$ with respect to $\mu_N$, where \[ H(\sigma)=\sum_{a\le M}U^\xi(S_a(\sigma))+P(\sigma) +\frac n{2N'}\sum_{a\le M}U''(S_a(\sigma)),\qquad\xi=\xi_{N'}, \] $P(\sigma)$ is the Gaussian part of the perturbation of the $N'$-dimensional system evaluated at $\rho(\sigma,0)$, and the last sum is the tilt $nB_N/2$. The row $g_a$ enters $H$ through $S_a=g_a\cdot\sigma/\sqrt{N'}$ and, in $P$, through the response coordinate $U'(x_a)/\sqrt{C_XN'}$ with $x_a=g_a\cdot\sigma/\sqrt N$. Hence $\partial_{g_{ai}}H(\sigma)=(\sigma_i/\sqrt{N'})\,\phi_a(\sigma)$ with \begin{align*} \phi_a&=(U^\xi)'(S_a)+\frac n{2N'}U'''(S_a)+\psi_a,\\ \psi_a&=\sqrt{\frac{N'}N}\,\frac{U''(x_a)}{\sqrt{C_XN'}}\, s_{N'}\sum_{v,m}c_{v,m}u_{v,m}\bigl[D\varpi(\Psi_{N'})^{\mathsf T} \partial_yh_{v,m}\bigr]_a , \end{align*} all evaluated at $\sigma$ and $\rho(\sigma,0)$. Since $S_aU'(S_a)=N'^{-1/2}\sum_ig_{ai}\sigma_iU'(S_a)$, Gaussian integration by parts in $g_{ai}$, conditionally on all other variables, gives \[ \E\langle S_aU'(S_a)\rangle_{\rm ref} =\frac N{N'}\,\E\Bigl[\langle U''(S_a)\rangle_{\rm ref} +\langle U'(S_a)\phi_a\rangle_{\rm ref} -\bigl\langle R_N(\sigma^1,\sigma^2)\,U'(S_a(\sigma^1))\, \phi_a(\sigma^2)\bigr\rangle_{\rm ref}\Bigr]. \] The first term comes from differentiating $U'(S_a)$, and the other two from differentiating the Gibbs density; we used $\sum_i\sigma_i^2=N$ and $\sum_i\sigma^1_i\sigma^2_i=NR_N(\sigma^1,\sigma^2)$. We justify the integration by parts. Write $h_{v,m}(\rho)=H_{v,m}(\rho,\widehat\Psi_{N'}(\rho))$, where $H_{v,m}(\rho,y)=\langle T_{v,m},F_{v,m}(\rho,y)\rangle$ and $F_{v,m}(\rho,y)$ is~\eqref{I:lo:tensor-field} with $\widehat\Psi_d(\rho)$ replaced by $y$. For $\rho\in S_{N'}$ and $\norm y\le1$, the derivative $\partial_yH_{v,m}(\rho,y)$ has norm at most $\mathcal I_{v,m}=m\sum_i\norm{T_{v,m,i}}_{\rm inj}$, as in Step~1 of the previous proof. By~\eqref{I:eq:lo-tensor} and Minkowski's inequality, $\mathcal I=\sum_{v,m}c_{v,m}\mathcal I_{v,m}$ is square integrable, and it does not depend on the rows. Since $\widehat\Psi_{N'}$ takes values in the unit ball, $D\varpi$ is bounded, $\abs{U''}\le L$, and $\abs{\sigma_i}\le\sqrt N$, the derivative in $g_{ai}$ of the term of index $(v,m)$ of $P$ is bounded by $C_Ns_{N'}c_{v,m}\mathcal I_{v,m}$, uniformly in $\sigma$ and $g_{ai}$. By the Weierstrass M-test, $P$ is continuously differentiable in $g_{ai}$, jointly continuous with its derivative in $(\sigma,g_{ai})$, and its derivative is the termwise one; the other two parts of $H$ are $C^1$ in $g_{ai}$ since $U\in C^4$. Differentiating under the integral over the compact sphere $S_N$ shows that $\mathsf F(g_{ai})=\langle\sigma_iU'(S_a)\rangle_{\rm ref}$, with all other variables fixed, is $C^1$ with the derivative used above. By $\abs{\sigma_i}\le\sqrt N$, $\abs{S_a}\le\norm{g_a}$, and the growth bounds in~\eqref{I:lo:class}, $\abs{\mathsf F}+\abs{\mathsf F'}\le C_N(1+\norm{g_a})^2(1+\mathcal I)$, which is integrable because $\mathcal I$ is independent of the rows. Hence Stein's lemma $\E[g\mathsf F(g)]= \E[\mathsf F'(g)]$ for a standard Gaussian $g$, valid for absolutely continuous $\mathsf F$ with $\E\abs{\mathsf F'(g)}<\infty$, applies conditionally on the other variables, and Fubini's theorem gives the display. Summing over $a\le M$, dividing by $N$, and writing $\phi_a=U'(S_a)+e_a$, the definitions in~\eqref{I:lo:observables} give \begin{multline*} \E\langle b_N\rangle_{\rm ref}-1-\E\langle X_N(\sigma,\sigma)\rangle_{\rm ref} +\E\langle R_N(\sigma^1,\sigma^2)X_N(\sigma^1,\sigma^2)\rangle_{\rm ref}\\ =\frac1{N'}\sum_{a\le M}\E\Bigl[\langle U'(S_a)e_a\rangle_{\rm ref} -\bigl\langle R_N(\sigma^1,\sigma^2)\,U'(S_a(\sigma^1))\, e_a(\sigma^2)\bigr\rangle_{\rm ref}\Bigr]. \end{multline*} Since $\abs{R_N}\le1$, the Cauchy--Schwarz inequality over $a$ bounds the right side by \[ \frac2{N'}\,\E\Bigl[\sup_\sigma\norm{U'(S(\sigma))}\, \sup_\sigma\norm{e(\sigma)}\Bigr], \] with Euclidean norms over $a\le M$. We bound the factors separately, using $\sum_aS_a^2\le\norm\bG_{\rm op}^2$ and $\E\norm\bG_{\rm op}^2\le C(M+N)$ (\Cref{I:lo:opnorm}). \begin{enumerate}[label=\textup{(\alph*)}] \item Since $\abs{U'(x)}\le L(1+\abs x)$, $\sup_\sigma\norm{U'(S(\sigma))}^2\le2L^2(M+\norm\bG_{\rm op}^2)$. \item The scalar perturbation contributes $\xi(u_1\tau_1'+u_2\tau_2')(S_a)$ to $e_a$. Since $\tau_1'=U'+xU''-U'''$ and $\tau_2'=2U'U''$ grow at most linearly, its norm is at most $C\xi(M+\norm\bG_{\rm op}^2)^{1/2}$. \item The tilt contributes $\frac n{2N'}U'''(S_a)$, of norm at most $nL\sqrt M/(2N')$. \item The Gaussian part contributes $\psi_a$. Since $D\varpi$ is bounded, $u_{v,m}\le5/2$ and $\abs{U''}\le L$, we have $\norm\psi\le Cs_{N'}N^{-1/2}\sum_{v,m}c_{v,m}\sup_\rho\norm{\partial_yh_{v,m}(\rho)}$. By~\eqref{I:eq:lo-feature-derivative} and the Cauchy--Schwarz inequality with the summable weights $c_{v,m}$, $\E\sup_\sigma\norm{\psi(\sigma)}^2\le Cs_{N'}^2(N'+M)/N$. \end{enumerate} Since $M\le2\alpha N$, the right side is at most $CN^{-1}(CN)^{1/2}(\xi_{N'}^2N+n^2/N+s_{N'}^2)^{1/2}\le C_n(\xi_{N'}+N^{-1}+s_{N'}N^{-1/2})$, uniformly in the parameters and in $M\in W_N$. This tends to zero, since $\xi_{N'}=N'^{-1/4}$ and $s_{N'}=N'^{1/3}$. \end{proof} \begin{proof}[Proof of \Cref{I:lo-cont}] We divide the proof into four steps. \proofstep{1}{Reduction to the arrays.} Let $\mathsf S$ be the closed set of arrays in which the $R$- and $X$-blocks are positive semidefinite; all arrays of the random measures and their limits lie in $\mathsf S$. By~\ref{I:lo:G}, for every $r$ and every bounded measurable function $\Phi$ of the arrays of $r$ replicas and of the values of the fields at these replicas, $\E\langle\Phi\rangle=\E\langle\bar\Phi(\mathbf A_r)\rangle$. If $\Phi$ is continuous, then $\bar\Phi$ is continuous on $\mathsf S$, since centered Gaussian laws depend continuously on their covariance matrices. Hence $\E\langle\Phi\rangle$ converges under convergence of the finite-dimensional distributions of the arrays, and its limit depends only on the limiting law. \proofstep{2}{Moment bounds.} Write $W(\rho)=\int_{\mathbb B_\Lambda}\nu_n(\dd\varepsilon) \e^{\varepsilon\cdot z(\rho)-(\norm\varepsilon^2-n)(\mathsf b(\rho)-1)/2}$, so that $\mathcal N_\Lambda=\langle W\rangle$. On the fixed ball, $W\le\exp\{\sqrt{\Lambda n}\norm{z(\rho)}+C_{n,\Lambda}(1+\abs{\mathsf b(\rho)})\}$, and Jensen's inequality for the normalized restriction of $\nu_n$ to the centered ball gives $W\ge\nu_n(\mathbb B_\Lambda)\exp\{-C_{n,\Lambda}(1+\abs{\mathsf b(\rho)})\}$. Under $\mathsf N_{\mathbf a}$, $z(\rho^1)$ has the law $N(0,X_{11}I_n)$, and $\E\e^{c\norm z}\le2^n\e^{nc^2X_{11}/2}$ for such a vector. By~\ref{I:lo:G}, applied to truncations and by monotone convergence, and by the moment hypothesis, $\E\langle W^q\rangle$ and $\E\langle W^{-q}\rangle$ are bounded along the sequence for every fixed $q\ge1$. By Jensen's inequality, $\langle W\rangle^{\pm q}\le\langle W^{\pm q}\rangle$, so $\E[\mathcal N_\Lambda^q+\mathcal N_\Lambda^{-q}]$ is bounded as well. The same holds for $\mathcal D=\langle\e^{y}\rangle$, since $y(\rho^1)$ has the law $N(0,nX_{11})$ under $\mathsf N_{\mathbf a}$. \proofstep{3}{Parts (i) and (ii).} Let $T\ge1$, put $W_T=\min\{W,T\}$ and $\mathcal N_T=\langle W_T\rangle$. Then $0\le\log\mathcal N_\Lambda-\log\mathcal N_T\le(\mathcal N_\Lambda-\mathcal N_T)/\mathcal N_T$, with $\mathcal N_\Lambda-\mathcal N_T\le\langle W^2\rangle/T$ and $\mathcal N_T\ge\langle\min\{W,1\}\rangle$, whose negative moments are bounded by Step~2. By the Cauchy--Schwarz inequality, $\E\abs{\log\mathcal N_\Lambda-\log\mathcal N_T}\le C/T$ uniformly along the sequence. Next let $0<\delta<1$, and let $P$ be a polynomial with $\abs{P(x)-\log\max\{x,\delta\}}\le\delta$ on $[0,T]$. Since $\mathcal N_T\in(0,T]$ and $\Prob(\mathcal N_T<\delta)\le\delta^2\E \mathcal N_T^{-2}$, the moment bounds of Step~2 give \[ \E\abs{\log\mathcal N_T-P(\mathcal N_T)} \le\delta+\E\bigl[(\abs{\log\mathcal N_T}+\abs{\log\delta}) \1_{\{\mathcal N_T<\delta\}}\bigr]\le\delta+C\delta\abs{\log\delta}, \] uniformly along the sequence. Finally, $\E[\mathcal N_T^j]=\E\langle \prod_{\ell\le j}W_T(\rho^\ell)\rangle$ is the expectation of a bounded continuous function of the arrays of $j$ replicas and of the values of $z$ at them, so it converges by Step~1, with a limit that depends only on the limiting law. Letting $N\to\infty$, then $\delta\to0$ and $T\to\infty$, proves the assertion of~(i) for $\mathcal N_\Lambda$; the denominator $\mathcal D$ is treated in the same way. For~(ii), the expectation of a bounded continuous function $F$ of the arrays and the cavity coordinates of $r$ replicas under the cavity measure is $\E[\langle\Phi_F\rangle/ \mathcal N_\Lambda^r]$, where \[ \Phi_F=\int_{\mathbb B_\Lambda^r}F\prod_{\ell\le r} \e^{\varepsilon^\ell\cdot z(\rho^\ell)-(\norm{\varepsilon^\ell}^2-n) (\mathsf b(\rho^\ell)-1)/2}\nu_n(\dd\varepsilon^\ell) \] satisfies $\abs{\Phi_F}\le\norm F_\infty\prod_\ell W(\rho^\ell)$. Let $\Phi_F^T$ be defined in the same way with each factor multiplied by $W_T(\rho^\ell)/W(\rho^\ell)$; it is a bounded continuous function of the arrays and of the values of $z$ at the $r$ replicas, and $\abs{\Phi_F^T}\le\norm F_\infty\prod_\ell W_T(\rho^\ell)$. Replacing one factor at a time gives $\abs{\langle\Phi_F\rangle-\langle\Phi_F^T\rangle}\le r\norm F_\infty\mathcal N_\Lambda^{r-1}(\mathcal N_\Lambda-\mathcal N_T)$, and $\abs{\mathcal N_\Lambda^{-r}-\mathcal N_T^{-r}}\langle\abs{\Phi_F^T}\rangle \le r\norm F_\infty(\mathcal N_\Lambda-\mathcal N_T)/\mathcal N_T$. As above, both errors are at most $C/T$ in expectation, uniformly along the sequence. Finally, $\abs{\langle\Phi_F^T\rangle}/\mathcal N_T^r\le\norm F_\infty$, and replacing $x\mapsto x^{-r}$ by a polynomial that approximates $\max\{x,\delta\}^{-r}$ within $\delta$ on $[0,T]$ costs at most $\norm F_\infty(T^r\delta+3\Prob(\mathcal N_T<\delta))$ in expectation. The expectations $\E[\langle\Phi_F^T\rangle\mathcal N_T^j]=\E\langle\Phi_F^T(\rho^1,\dots,\rho^r) \prod_{\ell=r+1}^{r+j}W_T(\rho^\ell)\rangle$ converge by Step~1, and letting $N\to\infty$, then $\delta\to0$ and $T\to\infty$ proves~(ii). \proofstep{4}{Part (iii).} A quadratic lower bound on $U$ suffices; negative exponential moments of $U$ are not needed. Fix $\pi$ and $T>0$, and put $U_T=\max\{U,-2T\}$. Apply \Cref{I:lo-trunc} with $V_1=\sum_{i\le\pi}U(y_i)$, $V_2=\sum_{i\le\pi}U_T(y_i)$, $A=\pi U_{\max}^+$, and $Y=C_0\sum_{i\le\pi}(1+y_i^2)$; by~\ref{I:lo:G} and $R(\rho,\rho)=1$, each $y_i(\rho)$ is standard Gaussian for one replica, so $\E[\langle Y\rangle^2]\le\E\langle Y^2\rangle\le6C_0^2\pi^2$. Since $0\le\e^{U_T}-\e^U\le\e^{-2T}$ and all activations are at most $U_{\max}^+$, writing $\e^{V_2}-\e^{V_1}$ as a telescoping sum over the rows gives $\e^{V_2}-\e^{V_1}\le\pi\e^{(\pi-1)U_{\max}^+}\e^{-2T}$. Hence \[ 0\le\E\log\langle\e^{V_2}\rangle-\E\log\langle\e^{V_1}\rangle \le\pi\e^{(\pi-1)U_{\max}^+}\e^{-T}+\frac{C\pi^2}T \] uniformly over the random measures. For fixed $T$ and $\pi$ the weight $\e^{V_2}$ lies between $\e^{-2\pi T}$ and $\e^{\pi U_{\max}^+}$ and is a bounded continuous function of the values of $y_1,\dots,y_\pi$, so the argument of Step~3, with $\log$ approximated by polynomials on this compact interval, applies to it. Letting $T\to\infty$ proves~(iii) for a fixed number of rows. For a Poisson number of rows, independent of the other variables, the bounds above are integrable in $\pi$, and $\abs{\E\log\langle\e^{V_1}\rangle}\le C(1+\pi)$ given $\pi$, so dominated convergence applies. Finally, if $\E\langle\abs{\mathsf b-\bar b_N}\rangle\to0$ and $\bar b_N\to b_\infty$, then the arrays with $\mathsf b$ and with $\mathsf b$ replaced by $b_\infty$ have the same limiting finite-dimensional distributions, and the random measures with the constant $b_\infty$ satisfy the hypotheses of the lemma. Since the limits depend only on the limiting law, $\mathsf b$ may be replaced by $b_\infty$. \end{proof} \chapter{Proof of the Parisi formula}\label{I:sec:parisi} In this chapter we prove \Cref{I:thm:parisi}: for $U$ in class~\ref{I:class-A} or class~\ref{I:class-B}, $F_N(U)\to\cP_*(U)$ and $f_N(U)\to\cP_*(U)$ in probability, the infimum defining $\cP_*(U)$ is attained, and every minimizer $\gamma$ satisfies \begin{equation}\label{I:pp:rU} Q_\gamma\le r_U,\qquad r_U=\frac{2\alpha C_U}{1+2\alpha C_U}, \end{equation} where $C_U$ is the constant of \Cref{I:lo-phi-cont}. In class~\ref{I:class-A}, $C_U\le C^{\rm A}_U=\norm{U'}_\infty^2/2$, and since $x\mapsto x/(1+x)$ is increasing, $r_U\le\alpha\norm{U'}_\infty^2/(1+\alpha\norm{U'}_\infty^2)$, as stated in \Cref{I:thm:parisi}. We combine the upper bound of \Cref{I:sec:upper} (\Cref{I:up-soft}, which applies to every activation of class~\ref{I:class-A}) with the lower bound of \Cref{I:sec:lower} (\Cref{I:lo-lower}). The lower bound was proved under the additional smoothness~\eqref{I:lo:class}; \Cref{I:pp:mollify} removes it by mollification. The upper bound requires a bounded first derivative, which may fail in class~\ref{I:class-B}; there we approximate $U$ from above by the activations $-T+\log(1+\e^{U+T})$, which belong to class~\ref{I:class-A}. Convergence in probability follows from the Gaussian Poincar\'e inequality and the operator-norm bound of \Cref{I:lo:opnorm}. Finally, the truncation lemma (\Cref{I:pp:truncation}) reduces the infimum to the compact class $\cU_{r_U}$, on which $\cP_U$ is continuous, and places every minimizer in $\cU_{r_U}$. \section{Mollification}\label{I:pp:mollification} Fix an even smooth probability density $\vartheta$ on $\R$ supported in $[-1,1]$, and for $0<\epsilon\le1$ put $\vartheta_\epsilon(x)=\epsilon^{-1}\vartheta(x/\epsilon)$ and $U_\epsilon=U*\vartheta_\epsilon$. \begin{lemma}\label{I:pp:mollify} Let $U$ belong to class~\ref{I:class-A} or class~\ref{I:class-B}, and put $\delta_\epsilon=\epsilon^2\norm{U''}_\infty/2$. \begin{enumerate}[label=\textup{(\alph*)}] \item $U_\epsilon$ satisfies~\eqref{I:lo:class} and belongs to the same class as $U$. \item $\abs{U_\epsilon(x)-U(x)}\le\delta_\epsilon$ for all $x\in\R$. \item For every $N$, pointwise in the disorder, $\abs{f_N(U_\epsilon)-f_N(U)}\le\alpha_N\delta_\epsilon$, where $\alpha_N=M_N/N$. For every $\gamma\in\cU$, $\abs{\cP_{U_\epsilon}(\gamma)-\cP_U(\gamma)}\le\alpha\delta_\epsilon$; consequently $\abs{\cP_*(U_\epsilon)-\cP_*(U)}\le\alpha\delta_\epsilon$. \end{enumerate} \end{lemma} \begin{proof} We prove the three parts in order. \proofpart{Proof of (a).} The function $U_\epsilon$ is smooth, and for $2\le j\le4$ its derivative $U_\epsilon^{(j)}=U''*\vartheta_\epsilon^{(j-2)}$ is bounded by $\norm{U''}_\infty\norm{\vartheta_\epsilon^{(j-2)}}_{L^1}$; these bounds depend on $\epsilon$, which is allowed in~\eqref{I:lo:class}. Since $\vartheta_\epsilon$ is a probability density supported in $[-1,1]$, we have $\sup U_\epsilon\le\sup U$, $\norm{U_\epsilon''}_\infty\le \norm{U''}_\infty$, $\norm{U_\epsilon'}_\infty\le\norm{U'}_\infty$, and $\abs{U_\epsilon'(x)}\le\sup_{\abs y\le1}\abs{U'(x-y)}\le\abs{U'(0)} +\norm{U''}_\infty(1+\abs x)$. Convolution with a nonnegative kernel preserves concavity. Hence $U_\epsilon$ belongs to the class of $U$ and satisfies~\eqref{I:lo:class}, with $L$ chosen large in terms of $\abs{U_\epsilon(0)}$, $\abs{U'(0)}$, $\norm{U''}_\infty$, and $\epsilon$. \proofpart{Proof of (b).} By Taylor's formula, $U(x-y)=U(x)-U'(x)y+\frac12U''(\bar x)y^2$ for some $\bar x$ between $x$ and $x-y$. Integrating against $\vartheta_\epsilon(y)\dd y$, the linear term vanishes because $\vartheta_\epsilon$ is even, and the quadratic term is at most $\frac12\norm{U''}_\infty\int y^2\vartheta_\epsilon(y)\dd y\le \delta_\epsilon$. \proofpart{Proof of (c).} By (b), $\e^{-M_N\delta_\epsilon}\le Z_N(U_\epsilon)/Z_N(U)\le \e^{M_N\delta_\epsilon}$, which gives the first assertion. For the second, \eqref{I:pf:supcontract}, which holds for every $\gamma\in\cU$ since $U$ and $U_\epsilon$ belong to the same class by (a), and (b) give $\abs{u_\gamma(0,0;U_\epsilon)-u_\gamma(0,0;U)}\le\delta_\epsilon$. The entropy term does not depend on the activation, so $\abs{\cP_{U_\epsilon}(\gamma)-\cP_U(\gamma)}\le\alpha\delta_\epsilon$, and taking the infimum over $\gamma$ gives the last assertion. \end{proof} \section{Truncation near \texorpdfstring{$q=1$}{q=1}}\label{I:pp:truncation-sec} The functional $\cP_U$ is continuous on each compact class $\cU_r$, $r<1$, by~\eqref{I:lo-phi-lip}, but $\cU$ is not compact: mass of an order parameter can escape to $q=1$. When the mass above the level $r_U$ of~\eqref{I:pp:rU} is moved down to $r_U$, the energy term rises, measured against $(1-\gamma(s))\dd s$ on $[r_U,1]$, at rate at most $\alpha C_U$, and the entropy falls at rate at least $s/(2(1-s))$, which exceeds $\alpha C_U$ for $s>r_U$. The next lemma is the analogue for $\cP_U$ of a fact about the spherical mixed $p$-spin model, whose Parisi formula was proved in \cite{Talagrand2006Spherical,Chen2013}: its Parisi measures are supported in an interval $[0,q_1]$ with $q_1<1$ depending only on the model (see the discussion in \cite{Talagrand2006Spherical} and \cite[Section~5]{AuffingerChenProperties}). \begin{lemma}\label{I:pp:truncation} Let $U$ belong to class~\ref{I:class-A} or class~\ref{I:class-B}, and let $r_U$ be as in~\eqref{I:pp:rU}. For $\gamma\in\cU$ let $\widehat\gamma=\gamma$ on $[0,r_U)$ and $\widehat\gamma=1$ on $[r_U,1]$. Then $\widehat\gamma\in\cU_{r_U}$ and \begin{equation}\label{I:pp:truncation-gain} \cP_U(\widehat\gamma)-\cP_U(\gamma) \le-\int_{r_U}^1(1-\gamma(s))\Big(\frac s{2(1-s)}-\alpha C_U\Big)\dd s \le0 . \end{equation} The inequality $\cP_U(\widehat\gamma)\le\cP_U(\gamma)$ is strict if $Q_\gamma>r_U$. Consequently $\cP_*(U)=\inf_{\gamma\in\cU_{r_U}}\cP_U(\gamma)$, and every minimizer $\gamma$ of $\cP_U$ satisfies $Q_\gamma\le r_U$. \end{lemma} \begin{proof} Since $\gamma\le1$, the function $\widehat\gamma$ is nondecreasing and right-continuous, and it equals one on $[r_U,1]$; so $\widehat\gamma\in\cU_{r_U}$. Moreover $\widehat\gamma-\gamma=(1-\gamma)\1_{[r_U,1]}\ge0$, and $1-\gamma(s)=0$ for $s\ge Q_\gamma$. We divide the rest of the proof into three steps. \proofstep{1}{The energy term.} As distribution functions of probability measures on $[0,1]$, $\gamma$ and $\widehat\gamma$ satisfy $W_1(\gamma,\widehat\gamma)=\int_0^1 (\widehat\gamma-\gamma)\dd s=\int_{r_U}^1(1-\gamma(s))\dd s$, so \eqref{I:lo-phi-cont-eq} gives \[ \alpha u_{\widehat\gamma}(0,0;U)-\alpha u_\gamma(0,0;U) \le\alpha C_U\int_{r_U}^1(1-\gamma(s))\dd s . \] \proofstep{2}{The entropy term.} Integrating $\gamma\le\widehat\gamma\le1$ over $[q,1]$ gives $0<\lambda_\gamma(q)\le\lambda_{\widehat\gamma}(q)\le1-q$ for $q<1$, and \[ \lambda_{\widehat\gamma}(q)-\lambda_\gamma(q) =\int_{\max\{q,r_U\}}^1(1-\gamma(s))\dd s . \] The integrands in \eqref{I:pf:CScutofffree} for $\gamma$ and for $\widehat\gamma$ take values in $[0,(1-Q_\gamma)^{-1}]$, because $\lambda_{\widehat\gamma}\ge\lambda_\gamma\ge1-Q_\gamma$ on $[0,Q_\gamma]$, and they vanish on $[Q_\gamma,1]$. Subtracting the two instances of \eqref{I:pf:CScutofffree} and using $\lambda_\gamma\lambda_{\widehat\gamma}\le(1-q)^2$, \begin{align*} \CS(\gamma)-\CS(\widehat\gamma) &=\frac12\int_0^1\frac{\lambda_{\widehat\gamma}(q)-\lambda_\gamma(q)} {\lambda_\gamma(q)\lambda_{\widehat\gamma}(q)}\dd q \ge\frac12\int_0^1\frac{\lambda_{\widehat\gamma}(q)-\lambda_\gamma(q)} {(1-q)^2}\dd q\\ &=\frac12\int_{r_U}^1(1-\gamma(s))\frac s{1-s}\dd s . \end{align*} The last equality is Tonelli's theorem: the function $(1-\gamma(s))(1-q)^{-2}$ is nonnegative on the region $\{(q,s):0\le q\le s,\ r_U\le s<1\}$, and $\int_0^s(1-q)^{-2}\dd q=s/(1-s)$. \proofstep{3}{Conclusion.} Adding the two estimates gives the first inequality in~\eqref{I:pp:truncation-gain}. The function $s\mapsto s/(1-s)$ is increasing on $[0,1)$ and equals $2\alpha C_U$ at $s=r_U$, so the integrand in~\eqref{I:pp:truncation-gain} is nonnegative, and it is positive at every $s\in(r_U,Q_\gamma)$, where $\gamma(s)<1$. If $Q_\gamma>r_U$, this interval has positive length and the inequality is strict. Since $\cU_{r_U}\subset\cU$, we obtain $\cP_*(U)=\inf_{\cU_{r_U}}\cP_U$. If $\gamma$ is a minimizer with $Q_\gamma>r_U$, then $\cP_U(\widehat\gamma)<\cP_U(\gamma)=\cP_*(U)$, which is impossible. Therefore every minimizer satisfies $Q_\gamma\le r_U$. \end{proof} \section{The two bounds and attainment of the infimum}\label{I:pp:proof} \begin{proof}[Proof of \Cref{I:thm:parisi}] Let $U$ belong to class~\ref{I:class-A} or class~\ref{I:class-B}. We divide the proof into four steps: the lower bound, the upper bound, convergence in probability, and attainment together with~\eqref{I:pp:rU}. \proofstep{1}{The lower bound.} Let $0<\epsilon\le1$. By \Cref{I:pp:mollify}(a), $U_\epsilon$ satisfies~\eqref{I:lo:class} and belongs to the class of $U$, so $U_\epsilon'$ is bounded or $U_\epsilon$ is concave; hence \Cref{I:lo-lower} gives $\liminf_NF_N(U_\epsilon)\ge\cP_*(U_\epsilon)$. By \Cref{I:pp:mollify}(c) and $\alpha_N\to\alpha$, \[ \liminf_{N\to\infty}F_N(U)\ge\liminf_{N\to\infty}F_N(U_\epsilon) -\alpha\delta_\epsilon\ge\cP_*(U_\epsilon)-\alpha\delta_\epsilon \ge\cP_*(U)-2\alpha\delta_\epsilon . \] Letting $\epsilon\to0$ gives $\liminf_NF_N(U)\ge\cP_*(U)$. The limits are taken in the order: first $\epsilon$ is fixed and $N\to\infty$ (inside \Cref{I:lo-lower}, the cavity dimension $n$ is sent to infinity after $N$), then $\epsilon\to0$. \proofstep{2}{The upper bound.} If $U$ belongs to class~\ref{I:class-A}, \Cref{I:up-soft} gives $\limsup_NF_N(U)\le\cP_U(\gamma)$ for every $\gamma\in\cU$, hence $\limsup_NF_N(U)\le\cP_*(U)$. Let now $U$ belong to class~\ref{I:class-B}, and put $B=\sup U$ and $K=\norm{U''}_\infty$. If $K=0$, then $U$ is affine and bounded above, hence constant, and it belongs to class~\ref{I:class-A}. Assume $K>0$. We first show that $\abs{U'}$ is controlled by the distance of $U$ from its supremum: \begin{equation}\label{I:pp:slope} U'(x)^2\le2K\,(B-U(x)),\qquad x\in\R . \end{equation} Indeed, Taylor's formula gives $B\ge U(x+y)\ge U(x)+U'(x)y-Ky^2/2$ for every $y\in\R$, and the choice $y=U'(x)/K$ gives $B\ge U(x)+U'(x)^2/(2K)$. For $T>0$ let $\mathsf s(v)=\e^v/(1+\e^v)$ and \[ U_T(x)=-T+\log\bigl(1+\e^{U(x)+T}\bigr). \] Then $U_T\ge U$, since $\log(1+\e^v)\ge v$; $U_T$ decreases as $T$ increases, since $\partial_TU_T=\mathsf s(U+T)-1<0$; and $U_T\downarrow U$ pointwise, since $U_T-U=\log(1+\e^{-U-T})$. Moreover $U_T$ belongs to class~\ref{I:class-A}. It is $C^2$ with $\sup U_T\le-T+\log(1+\e^{B+T})$, and its derivatives are \[ U_T'=\mathsf s(U+T)\,U',\qquad U_T''=\mathsf s(U+T)\,U''+\mathsf s'(U+T)\,(U')^2 . \] By~\eqref{I:pp:slope}, with $v=U(x)+T\le B+T$, \[ \abs{U_T'(x)}^2\le2K\,\mathsf s(v)^2(B+T-v),\qquad \abs{U_T''(x)}\le K+2K\,\mathsf s'(v)(B+T-v). \] Since $\mathsf s(v)\le\e^v$ and $0\le\mathsf s'(v)\le\e^v$, both right sides are bounded uniformly in $v\le B+T$. Hence $U_T'$ and $U_T''$ are bounded. Fix a step order parameter $\gamma$ and $T>0$. Since $U\le U_T$, we have $Z_N(U)\le Z_N(U_T)$, and \Cref{I:up-soft} applied to $U_T$ gives \[ \limsup_{N\to\infty}F_N(U)\le\limsup_{N\to\infty}F_N(U_T)\le\cP_{U_T}(\gamma). \] We now let $T\to\infty$. The quantity $u_\gamma(0,0;U_T)$ is computed by the recursion~\eqref{I:pf:recursion} started from the terminal datum $U_T$, and we show by induction over the steps that the intermediate functions decrease pointwise to those computed from $U$; they decrease in $T$ by~\eqref{I:pf:monotone}. At a step with positive parameter $m$, the functions $\e^{mh_T}$ decrease to $\e^{mh}$ and are bounded by $\e^{m\sup h_{T_0}}$ for $T\ge T_0$, so monotone convergence applies. At a step with parameter zero, the functions $h_T$ are bounded above by $\sup h_{T_0}$ and below by $h$, and $h(x)\ge-C(2+x^2)$ by~\eqref{I:pf:quadlower}, since $U(x)\ge U(0)+U'(0)x-Kx^2/2\ge-C(1+x^2)$. Dominated convergence therefore applies. Hence $u_\gamma(0,0;U_T)\downarrow u_\gamma(0,0;U)$ and $\cP_{U_T}(\gamma)\downarrow\cP_U(\gamma)$, so \[ \limsup_{N\to\infty}F_N(U)\le\cP_U(\gamma) \] for every step order parameter $\gamma$. Finally let $\gamma\in\cU$ be arbitrary, and fix $\bar q\in[Q_\gamma,1)$. The step order parameters $\gamma_j$ of \Cref{I:pf:step} with $\gamma_j=1$ on $[\bar q,1]$ and $\norm{\gamma_j-\gamma}_{L^1}\to0$ satisfy $\cP_U(\gamma_j)\to\cP_U(\gamma)$ by~\eqref{I:lo-phi-lip} with $r=\bar q$. Therefore $\limsup_NF_N(U)\le\cP_U(\gamma)$ for every $\gamma\in\cU$, that is, $\limsup_NF_N(U)\le\cP_*(U)$. The order of limits is: first the step order parameter $\gamma$ and $T$ are fixed and $N\to\infty$; then $T\to\infty$; then $\gamma$ is approximated. Combining Steps~1 and~2, $\lim_NF_N(U)=\cP_*(U)$ in both classes. \proofstep{3}{Convergence in probability.} We apply the Gaussian Poincar\'e inequality (\Cref{I:sec:lower-technical}) to $f_N(U)$ as a function of the entries $g_{ai}$ of $\bG$; it is continuously differentiable, with $\partial_{g_{ai}}f_N(U)=N^{-3/2}\langle U'(g_a\cdot\sigma/\sqrt N)\sigma_i\rangle$, where $\langle\cdot\rangle$ is the Gibbs measure with density proportional to $\exp\{\sum_aU(g_a\cdot\sigma/\sqrt N)\}$. By the Cauchy--Schwarz inequality and $\norm\sigma^2=N$, $\sum_i\langle U'(g_a\cdot\sigma/\sqrt N)\sigma_i\rangle^2\le N\langle U'(g_a\cdot\sigma/\sqrt N)^2\rangle$, so \[ \Var(f_N(U))\le\frac1{N^2}\, \E\Big\langle\sum_{a\le M_N}U'(g_a\cdot\sigma/\sqrt N)^2\Big\rangle . \] In class~\ref{I:class-A} the right side is at most $\alpha_N\norm{U'}_\infty^2/N$. In class~\ref{I:class-B}, $\abs{U'(x)}\le\abs{U'(0)}+\norm{U''}_\infty\abs x\le C(1+\abs x)$ and $\sum_a(g_a\cdot\sigma)^2/N\le\norm\bG_{\rm op}^2$, and \Cref{I:lo:opnorm} gives $\E\norm\bG_{\rm op}^2\le4(M_N+N)+8$. Hence the right side is at most $C(M_N+\E\norm\bG_{\rm op}^2)/N^2=O(1/N)$. In both cases $f_N(U)-F_N(U)\to0$ in probability by Chebyshev's inequality, and Steps~1 and~2 give $f_N(U)\to\cP_*(U)$ in probability. \proofstep{4}{Attainment and the location of minimizers.} By \Cref{I:pp:truncation}, $\cP_*(U)$ is the infimum of $\cP_U$ over $\cU_{r_U}$. The class $\cU_{r_U}$ is a compact metric space for the $L^1$ distance (\Cref{I:pf:step}), and $\cP_U$ is Lipschitz on it by~\eqref{I:lo-phi-lip}. A continuous real function on a compact metric space attains its infimum, so $\cP_U(\gamma_*)=\cP_*(U)$ for some $\gamma_*\in\cU_{r_U}$. By \Cref{I:pp:truncation}, every minimizer satisfies~\eqref{I:pp:rU}. This proves \Cref{I:thm:parisi}. \end{proof} \chapter{The variational problem at hard margins}\label{I:sec:hardwall} This chapter studies the deterministic minimization problem $\cP_*(\kappa)=\inf_{\gamma\in\cU}\cP_\kappa(\gamma)$ of \eqref{I:eq-kappac} and proves \Cref{I:thm:jamming}; Step~3 of \Cref{I:sec:ideas} outlines the argument. No random constraints appear; the only random objects are the Parisi diffusion and auxiliary Gaussian variables. \Cref{I:hw:value-sec} proves the value identity and the existence of minimizers, \Cref{I:hw:slope-sec} proves the slope bound, and \Cref{I:hw:top-sec} proves weak top stability, the replica symmetry of constrained minimizers at very negative margins, and the finiteness of $\kc$. \Cref{I:hw:critical-sec} proves the divergence of $\cP_*$ at $\kc$ and deduces the replica symmetry of minimizers at very negative margins. \Cref{I:hw:pin-sec} shows that a very negative value pins the top of every minimizer close to one, and deduces the collapse of minimizers at $\kc$. \Cref{I:sec:gardner} uses the definition of $\kc$ and the Gaussian bounds of \Cref{I:hw:gauss}; \Cref{I:sec:hard-volume} uses the results below to identify the hard-wall volume and the largest margin, and \Cref{I:sec:gaps} uses the collapse of minimizers to identify the gap law. The value identity (\Cref{I:fun:value}), the existence of minimizers (\Cref{I:fun:existence}), the slope bound (\Cref{I:fun:slope}), weak top stability (\Cref{I:fun:top}), the finiteness of $\kc$ (\Cref{I:fun:finite}) and the divergence at $\kc$ (\Cref{I:fun:critical}) are also used in Part~II, whose \Cref{II:fun:sec:value,II:fun:sec:thresholds} state them and refer to the proofs given here (Part~II, \Crefrange{II:lem:value}{II:lem:weaktop}, \Cref{II:fun:existence}, \Cref{II:fun:finite} and \Cref{II:fun:critical}). The stationarity conditions in \Cref{I:fun:value} and \eqref{I:hw:stat} are the analogues for $\cP_\kappa$ of the first-order and obstacle conditions for Parisi measures of mixed $p$-spin models, due to Talagrand \cite{TalagrandParisiMeasures}, to Jagannath and Tobasco \cite{JagannathTobasco2017}, and to W.-K.\ Chen \cite[Theorem~2 and Proposition~1]{Chen2017Variational}, and of their spherical versions, which appear in the work of Jagannath and Tobasco \cite{JagannathTobasco2017Spherical} and of Auffinger and W.-K.\ Chen \cite[Proposition~4]{AuffingerChenProperties}. Jagannath and Tobasco characterized Parisi measures as the measures carried by the minimum points of an antiderivative of the first variation \cite[Proposition~1.1]{JagannathTobasco2017}; there, convexity makes the condition also sufficient. For the spherical model, whose Crisanti--Sommers functional is the closer analogue of $\cP_\kappa$, the corresponding characterization is due to Talagrand \cite[Proposition~2.1]{Talagrand2006Spherical}. For $\cP_\kappa$ itself, El Alaoui and Sellke obtained $D=0$ and $S=1$ at a minimizer whose support is an interval, in directions supported on that interval \cite[Proposition~16]{ElAlaouiSellke2022}; see the opening of \Cref{I:sec:functional}. Here the conditions on $D$ (\Cref{I:fun:value} and \eqref{I:hw:stat}) hold for every minimizer over $\cU_r$, with no assumption on its support and including the case $Q=r$ of an active cutoff; the condition $S(Q)=1$ appears only at an accumulation point of the support (proof of \Cref{I:fun:top}). The value identity holds for every order parameter, and so does the slope bound, with a constant depending only on $\kappa$. These two results, weak top stability, the pinning of the top, and the collapse of minimizers at $\kc$ do not appear in these works. \paragraph{Conventions.} Throughout this chapter, $\alpha>0$ is fixed, and we use the notation of \Cref{I:fun:finite-cutoff}. For a margin $\kappa\in\R$ and $\gamma\in\cU$ we write \begin{gather*} Q=Q_\gamma,\qquad \delta=1-Q,\qquad \lambda=\lambda_\gamma,\qquad u=u_\gamma(\cdot,\cdot\,;\kappa),\\ b=\partial_xu,\qquad c=-\partial_{xx}u,\qquad v=\lambda c, \end{gather*} where $u$ is computed with the cutoff $Q$, so that $u(Q,x)=\log\Phi((x-\kappa)/\sqrt\delta)$ by \eqref{I:pf:hardonflat}. Thus $\lambda(q)=\delta+\int_q^Q\gamma$ for $q\le Q$, and in particular $\lambda\ge\delta$ on $[0,Q]$. The generator is $\cL=\partial_q+\frac12\partial_{xx}+\gamma b\,\partial_x$, the diffusion $X$ solves $\dd X_q=\gamma(q)b(q,X_q)\dd q+\dd B_q$ with $X_0=0$, and $D$ and $S$ are the functions in \eqref{I:fun:D}. For $r<1$, $\cU_r$ is the class of $\gamma\in\cU$ with $\gamma=1$ on $[r,1]$; for $\gamma\in\cU_r$ the functions $X$, $D$, and $S$ are defined on $[0,r]$. We write $\mu_\gamma$ for the probability measure on $[0,1]$ with distribution function $\gamma$; its support lies in $[0,Q]$. When the order parameter or the margin varies, we write $u_{\gamma,\kappa}$, $b_{\gamma,\kappa}$, and so on; thus $b_{\gamma,\kappa}(0,0)=\partial_xu_\gamma(0,0;\kappa)$. As in \Cref{I:fun:finite-cutoff}, $R=\phi/\Phi$, $\Rbar(z)=z+R(z)$, and $V=R\Rbar$; we also use \[ h(z)=\log\Phi(z)+\frac12R(z)^2 ,\qquad m(k)=\E(k-G)_+^2 , \] where $G$ denotes a standard Gaussian variable independent of $B$. Finally, the \emph{normalized top position} is \[ Z=\frac{X_Q-\kappa}{\sqrt\delta},\qquad\text{so that}\qquad u(Q,X_Q)=\log\Phi(Z),\qquad b(Q,X_Q)=\frac{R(Z)}{\sqrt\delta}. \] We use the following facts from \Cref{I:fun:finite-cutoff}. For each $r<1$ the class $\cU_r$ is compact in $L^1([0,1])$, and $\cP_\kappa$ is continuous on $\cU_r$, uniformly for $\kappa$ in compact intervals (\Cref{I:pf:coef-cont}); in particular $\cP_\kappa$ attains its minimum over $\cU_r$. By \eqref{I:fun:analytic}, $b>0$ and $00$ and, for $n\ge1$, put \begin{equation}\label{I:hw:stepapprox} \gamma_n(q)=\gamma(jQ/n)\quad\text{for }q\in[jQ/n,(j+1)Q/n),\ 0\le j0$ split each expectation according to whether $\abs{X^{\gamma_n}_q}\vee\abs{X_q}\le R$. On this event the integrands differ by at most a constant times the sum of $\sup_{[0,Q]\times[-R,R]}(\abs{b_{\gamma_n}-b}+\abs{c_{\gamma_n}-c})$ and $C_R\abs{X^{\gamma_n}_q-X_q}$, because the functions and their spatial derivatives are bounded on $[0,Q]\times[-R,R]$ uniformly in $n$. On the complement the integrands are bounded by $C(1+\abs{X^{\gamma_n}_q}+\abs{X_q})^p$, whose contribution is at most $C_pR^{-1}$ uniformly in $q$ and $n$ by the uniform moment bounds. Letting $n\to\infty$ and then $R\to\infty$ gives the uniform convergence. Since also $\lambda_{\gamma_n}\to\lambda$ uniformly with $\lambda_{\gamma_n}\ge\delta$, the functions $D$ of $\gamma_n$ converge to $D$ uniformly on $[0,Q]$. Since the top is fixed, the normalized top positions $Z_n$ of $\gamma_n$ converge to $Z$ in $L^2$; as $0z_-=\max\{-z,0\}$. \item $V$ is strictly decreasing. \item $V(z)^2\le2R(z)$ for every $z\in\R$. \item $h'=R(1-V)>0$ and $h<0$. \item With $C_0=-\log\Phi(-1)$, for every $t\in\R$, \begin{equation}\label{I:hw:tail} -\frac12t_+^2-\log(1+t_+)-C_0\le\log\Phi(-t)\le-\frac12t_+^2 . \end{equation} \item For every $t\in\R$ and $\theta>0$, with $\log_+y=\max\{\log y,0\}$, \begin{equation}\label{I:hw:tailshift} \log\Phi(-t)\ge-\frac12(t+\theta)_+^2-\log_+\frac1\theta-C_0 . \end{equation} \end{enumerate} \end{lemma} \begin{proof} The functions $R$, $\Rbar$, $V$, and $h$ are those of Part~II, \Cref{II:fun:sec:gauss}. \proofpart{Proof of (i)--(iv).} These are contained in Part~II, \Cref{II:fun:gauss}. Its part~(a) gives $R'=-V$, $00$, $\Rbar'=1-V>0$, and $R(z)>z_-$, which is (i), since $R'=-V<0$. Assertion (ii) is the inequality $V'<0$ in \eqref{II:fun:Vpp}, and assertions (iii) and (iv) are parts~(e) and~(d) of that lemma. \proofpart{Proof of (v).} The upper bound holds for $t\le0$ because $\Phi\le1$, and for $t>0$ it is the Chernoff bound $\Prob(G\ge t)\le\e^{-t^2}\E\e^{tG}=\e^{-t^2/2}$. For the lower bound, if $t\le1$ then $\log\Phi(-t)\ge\log\Phi(-1)=-C_0$. If $t\ge1$, the lower Mills bound of \Cref{I:pf:gauss}(i) and $2t^2\ge1+t^2$ give \[ \Phi(-t)>\frac{t\phi(t)}{1+t^2}\ge\frac{\phi(t)}{2t} \ge\frac{\e^{-t^2/2}}{2\sqrt{2\pi}\,(1+t)}, \] and $\log(2\sqrt{2\pi})<1.62<1.840$. Since $\frac12(t+\theta)^2\ge\frac12t^2+\theta t$, by (v) it suffices to show $\log(1+t)\le\theta t+\log_+(1/\theta)$. If $\theta\ge1$, then $\log(1+t)\le t\le\theta t$. If $\theta<1$, the concave function $t\mapsto\log(1+t)-\theta t$ is maximal at $t=1/\theta-1$, where it equals $\log(1/\theta)-1+\theta<\log(1/\theta)$. \end{proof} The value identity below involves the first-variation function $D$. At a minimizer over $\cU_r$, stationarity says that the antiderivative of $-D/2$ is minimal on the support of $\mu_\gamma$; this is the source of the sign information, as in the characterizations of Parisi measures recalled at the start of the chapter. In the proof, \eqref{I:hw:u00} is the control representation of Auffinger and W.-K.\ Chen \cite[Theorem~3]{AuffingerChen2015Uniqueness} and of Jagannath and Tobasco \cite[Lemma~18]{JagannathTobasco2016}, evaluated at its optimal control $b(q,X_q)$. \begin{lemma}\label{I:fun:value} For every $\kappa\in\R$ and $\gamma\in\cU$, \begin{equation}\label{I:fun:valueidentity} \cP_\kappa(\gamma)=\frac12\log\delta+\alpha\E h(Z)-\frac\delta2D(Q) -\frac12\int_0^Q\gamma(q)D(q)\dd q . \end{equation} If $r\in[0,1)$ and $\gamma$ minimizes $\cP_\kappa$ over $\cU_r$, then $\int_0^Q\gamma D\dd q=0$ and $D(Q)\ge0$, with $D(Q)=0$ if $Q0$. Both sides of \eqref{I:fun:valueidentity} converge along the approximations \eqref{I:hw:stepapprox}, so we may assume that $\gamma$ is a step order parameter. By It\^o's formula on each interval of constancy of $\gamma$, the equation \eqref{I:eq-pde} and the continuity of $u$, \begin{align*} \dd u(q,X_q)&=\Big(\partial_qu+\frac12\partial_{xx}u+\gamma b^2\Big)(q,X_q)\dd q +b(q,X_q)\dd B_q\\ &=\frac12\gamma(q)b(q,X_q)^2\dd q+b(q,X_q)\dd B_q . \end{align*} The stochastic integral has mean zero because $b(q,X_q)$ is square integrable. Since $u(Q,X_Q)=\log\Phi(Z)$, \begin{equation}\label{I:hw:u00} u(0,0)=\E\log\Phi(Z)-\frac12\int_0^Q\gamma(q)\E b(q,X_q)^2\dd q . \end{equation} By \eqref{I:fun:D}, $\alpha\E b(q,X_q)^2=D(q)+\int_0^q\lambda(t)^{-2}\dd t$. Multiplying by $\gamma(q)$, integrating over $[0,Q]$, exchanging the order of integration, and using $\int_t^Q\gamma=\lambda(t)-\delta$, we obtain \[ \alpha\int_0^Q\gamma(q)\E b(q,X_q)^2\dd q =\int_0^Q\gamma D\dd q+\int_0^Q\frac{\dd t}{\lambda(t)}-\delta\int_0^Q\frac{\dd t}{\lambda(t)^2}. \] At the terminal time $b(Q,X_Q)=R(Z)/\sqrt\delta$, so the definition of $D(Q)$ gives $\delta\int_0^Q\lambda^{-2}\dd t=\alpha\E R(Z)^2-\delta D(Q)$. Substituting these two identities into \eqref{I:hw:u00} and adding $\CS(\gamma)=\frac12\int_0^Q\lambda^{-1}\dd q+\frac12\log\delta$, the integral $\frac12\int_0^Q\lambda^{-1}$ cancels and we obtain \[ \cP_\kappa(\gamma)=\frac12\log\delta+\alpha\E\Big[\log\Phi(Z)+\frac12R(Z)^2\Big] -\frac\delta2D(Q)-\frac12\int_0^Q\gamma D\dd q, \] which is \eqref{I:fun:valueidentity}. Now let $\gamma$ minimize over $\cU_r$, and write $\mu=\mu_\gamma$. If $r=0$, then $\gamma\equiv1$, $Q=0$, and $D(0)=\alpha b(0,0)^2>0$, and the stationarity assertions hold; so let $r>0$. Put \[ \cH(t)=\cH_\gamma(t)=-\frac12\int_0^tD(q)\dd q\qquad(t\in[0,r]), \] so that $\cH'=-D/2$. For $t\in[0,r]$ the order parameter $\1_{[t,1]}$ belongs to $\cU_r$, and so does $\gamma+\theta(\1_{[t,1]}-\gamma)$ for $\theta\in[0,1]$. Minimality and \eqref{I:fun:variation} give $\int_0^r(\1_{[t,1]}-\gamma)D\dd q\ge0$. Here $\int_0^r\1_{[t,1]}D\dd q=\int_t^rD\dd q=2(\cH(t)-\cH(r))$, while, since $\gamma(q)=\mu([0,q])$, Fubini's theorem gives $\int_0^r\gamma D\dd q=\int\big(\int_s^rD(q)\dd q\big)\mu(\dd s)=2\int(\cH(s)-\cH(r))\mu(\dd s)$. Hence \begin{equation}\label{I:hw:Hmin} \cH(t)\ge\int\cH\dd\mu\qquad(0\le t\le r). \end{equation} Since $\mu$ is a probability measure on $[0,Q]\subset[0,r]$, \eqref{I:hw:Hmin} forces $\int\cH\dd\mu=\min_{[0,r]}\cH$, so $\cH$ equals its minimum $\mu$-almost everywhere and, being continuous, on $\supp\mu$. The point $Q$ belongs to $\supp\mu$: if $Q>0$, then $\gamma(q)<1$ for $q0$ for every $q0$ because $X_0=0$ and $b>0$. Since $r>0$, $\cH'(0)=-D(0)/2<0$, so $0$ is not a minimum point of $\cH$ on $[0,r]$, and $0\notin\supp\mu$; in particular $Q>0$. Since $Q$ is a minimum point of $\cH$ on $[0,r]$, $\cH(q)\ge\cH(Q)$ for $q0$, and $\cH$ is as in the proof (with $\cH'=-D/2$), then \begin{equation}\label{I:hw:stat} \supp\mu_\gamma\subset\argmin_{[0,r]}\cH,\qquad 0\notin\supp\mu_\gamma . \end{equation} For comparison, the order parameter $\gamma\equiv1$ has $Q=0$, $\delta=1$, $u(0,0)=\log\Phi(-\kappa)$ and $\CS(\gamma)=0$. Hence \begin{equation}\label{I:hw:test} \cP_*(\kappa)\le\cP_\kappa(1)=\alpha\log\Phi(-\kappa)<0\qquad(\kappa\in\R). \end{equation} For $r\in[0,1)$ we write \begin{equation}\label{I:hw:mr} \cP_*^r(\kappa)=\min_{\gamma\in\cU_r}\cP_\kappa(\gamma) \end{equation} for the constrained minimum. Since the classes $\cU_r$ increase with $r$ and their union is $\cU$, $\cP_*^r(\kappa)$ is nonincreasing in $r$ and decreases to $\cP_*(\kappa)$ as $r\uparrow1$. Also $\cP_*^0(\kappa)=\alpha\log\Phi(-\kappa)$, since $\cU_0=\{1\}$. \begin{theorem}\label{I:fun:existence} Let $\kappa\in\R$ with $\cP_*(\kappa)>-\infty$. Then the infimum defining $\cP_*(\kappa)$ is attained. Every minimizer $\gamma$ satisfies \[ 1-Q_\gamma>\e^{2\cP_*(\kappa)}, \] that is, every minimizer lies in $\cU_r$ with $r=1-\e^{2\cP_*(\kappa)}\in(0,1)$, and the set of minimizers is compact in $L^1([0,1])$. \end{theorem} \begin{proof} By \eqref{I:hw:test}, $-\infty<\cP_*(\kappa)<0$, so $r_*=1-\e^{2\cP_*(\kappa)}$ lies in $(0,1)$. For $r\in[0,1)$ let $\gamma_r$ minimize $\cP_\kappa$ over $\cU_r$, with top $Q_r$ and $\delta_r=1-Q_r$. By \eqref{I:fun:valueineq}, \[ \frac12\log\delta_r>\cP_\kappa(\gamma_r)\ge\cP_*(\kappa), \] so $Q_r\cP_\kappa(\gamma)=\cP_*(\kappa)$ by \Cref{I:fun:value}, which is the asserted bound; in particular $\gamma\in\cU_{r_*}$. The set of minimizers is $\{\gamma\in\cU_{r_*}:\cP_\kappa(\gamma)=\cP_*(\kappa)\}$, which is closed in the compact class $\cU_{r_*}$ by continuity, hence compact. \end{proof} The theorem concerns minimizers, and \eqref{I:fun:valueineq} gives the same bound for constrained minimizers. Neither says that the tops of an arbitrary minimizing sequence stay away from one: adding a vanishing mass near one changes the top without changing the $L^1$ limit. Like \Cref{I:pp:truncation}, the theorem is the analogue for $\cP_\kappa$ of the fact about spherical Parisi measures recalled before that lemma: they are supported in an interval $[0,q_1]$ with $q_1<1$. Here the bound on the top comes from the value identity, and it depends on $\cP_*(\kappa)$. \section{The slope in the margin}\label{I:hw:slope-sec} To move values between margins we need a bound on how fast $\cP_\kappa(\gamma)$ changes with $\kappa$. The following bound holds for every order parameter, which is what makes it usable for near-minimizers and for constrained minimizers alike. \begin{lemma}\label{I:fun:slope} Let $C_\kappa=\frac12\big(\kappa+\sqrt{\kappa^2+4}\big)$. For every $\gamma\in\cU$ and $\kappa\in\R$, \begin{equation}\label{I:hw:slopebound} \E(\kappa-B_Q)_+\le\lambda(0)\,b(0,0)0$, and first let $\gamma$ be a step order parameter. For smooth $f,g$ we have $\cL(fg)=f\cL g+g\cL f+\partial_xf\,\partial_xg$. On each interval of constancy of $\gamma$, \Cref{I:pf:apriori}(ii) gives $\cL b=0$ and $\cL c=\gamma c^2$; together with $\partial_xb=-c$, $\cL\tau=-1$, $\lambda'=-\gamma$, and $\cL(x-\kappa)=\gamma b$ we obtain \[ \cL(\tau c)=-c+\gamma\tau c^2,\qquad \cL(\lambda b^2)=-\gamma b^2+\lambda c^2,\qquad \cL\big((x-\kappa)b\big)=\gamma b^2-c . \] For instance, $\cL(b^2)=2b\,\cL b+(\partial_xb)^2=c^2$, and $\cL((x-\kappa)b)=(x-\kappa)\cL b+b\,\cL(x-\kappa)+\partial_xb$. Hence \[ \cL F=(\gamma\tau-\lambda)c^2\le0, \] because $\gamma$ is nondecreasing and therefore $\lambda(q)=\int_q^1\gamma\ge\gamma(q)(1-q)$. It\^o's formula on each interval of constancy gives $F(0,0)=\E F(Q,X_Q)-\E\int_0^Q\cL F(q,X_q)\dd q\ge0$. For general $\gamma$ the inequality $F(0,0)\ge0$ follows from the step approximations \eqref{I:hw:stepapprox}, since $b(0,0)$, $c(0,0)$ and $\lambda(0)$ converge along them. At $(0,0)$ this inequality reads $c(0,0)\ge\lambda(0)b(0,0)^2-\kappa b(0,0)$, and $\lambda(0)c(0,0)=v(0,0)<1$. Multiplying by $\lambda(0)>0$, the number $y=\lambda(0)b(0,0)>0$ satisfies $y^2-\kappa y<1$. Hence $y$ lies below the positive root $C_\kappa$ of $y^2-\kappa y-1$. For the lower bound write $X_Q=B_Q+\Delta$ with $\Delta=\int_0^Q\gamma(q)b(q,X_q)\dd q\ge0$. Since $b(q,X_q)$ is a martingale, $\E\Delta=b(0,0)\int_0^Q\gamma=b(0,0)(\lambda(0)-\delta)$. Since $(\kappa-X_Q)_+=\sqrt\delta\,Z_-$ and $R(z)\ge z_-$ (\Cref{I:hw:gauss}(i)), \eqref{I:hw:mart} gives $\E(\kappa-X_Q)_+\le\sqrt\delta\,\E R(Z)=\delta b(0,0)$. Finally $(\kappa-B_Q)_+\le(\kappa-X_Q)_++\Delta$, and adding the two expectations gives $\E(\kappa-B_Q)_+\le\lambda(0)b(0,0)$. The terminal datum depends on $\kappa$ only through $x-\kappa$, and the recursion \eqref{I:pf:recursion} commutes with translations in $x$, so $u_{\gamma,\kappa}(q,x)=u_{\gamma,0}(q,x-\kappa)$. By \Cref{I:pf:apriori}(v), which holds for every $\gamma\in\cU_r$ by \Cref{I:pf:coef-cont}, $u_{\gamma,0}(0,\cdot)$ has continuous derivatives of all orders. Therefore $\kappa\mapsto u_{\gamma,\kappa}(0,0)$ is twice continuously differentiable with $\partial_\kappa u_{\gamma,\kappa}(0,0)=-b(0,0)$ and $\partial_\kappa^2u_{\gamma,\kappa}(0,0)=-c(0,0)$, and $\partial_\kappa b_{\gamma,\kappa}(0,0)=c(0,0)>0$. Since $\CS(\gamma)$ does not depend on $\kappa$, $\partial_\kappa\cP_\kappa(\gamma)=-\alpha b(0,0)<0$ and $\partial^2_\kappa\cP_\kappa(\gamma)=-\alpha c(0,0)<0$. An infimum of nonincreasing concave functions is nonincreasing and concave. \end{proof} We use the lemma mostly through the tangent-line inequality: by concavity, for every $\gamma\in\cU$ and all margins $k,k'$, \begin{equation}\label{I:hw:tangent} \cP_{k'}(\gamma)\le\cP_k(\gamma)-\alpha\,b_{\gamma,k}(0,0)\,(k'-k). \end{equation} Recall from \eqref{I:eq-kappac} that $\kc=\sup\{\kappa:\cP_*(\kappa)>-\infty\}$. Since $\cP_*$ is nonincreasing, $\cP_*(\kappa)>-\infty$ for $\kappa<\kc$ and $\cP_*(\kappa)=-\infty$ for $\kappa>\kc$. \Cref{I:fun:finite} shows that $\kc$ is finite, and \Cref{I:fun:critical} shows that $\cP_*(\kc)=-\infty$. \section{Weak top stability and finiteness of the critical margin}\label{I:hw:top-sec} To show that $\kc>-\infty$ we must find margins with $\cP_*(\kappa)>-\infty$. For an order parameter with a single support point the value is explicit, so it suffices to show that, at very negative margins, constrained minimizers have a single support point. The next lemma gives a quantitative obstruction to a second support point, namely a lower bound on $S(Q)$, and a lower bound on the slope in terms of $S(Q)$. Together with the slope bound, these two bounds give $\lambda(0)\le C\sqrt\delta$ for a minimizer; this is used again in \Cref{I:fun:collapse-prop}. The proof follows the endpoint analysis of Auffinger and W.-K.\ Chen \cite[Proposition~3 and proof of Theorem~4]{AuffingerChenProperties}: when support points accumulate at $Q$, Rolle's theorem gives $S(Q)=1$. The differences are that both terms in \eqref{I:hw:c2} are nonnegative, because $c>0$, while the corresponding cubic term in \cite[Proposition~3]{AuffingerChenProperties} enters with a negative sign, and that an isolated top is handled by the quantitative inequality \eqref{I:hw:gapineq}. \begin{lemma}\label{I:fun:top} Let $\kappa\in\R$ and $r\in[0,1)$, and let $\gamma$ minimize $\cP_\kappa$ over $\cU_r$. If $\mu_\gamma$ has more than one support point, then $\alpha>1$ and \[ S(Q)\ge s_\alpha:=\min\Big\{\frac12,\frac1{16\alpha}\Big\}. \] Moreover, every $\gamma\in\cU$ satisfies, at every margin $\kappa$, \begin{equation}\label{I:hw:topslope} \sqrt\delta\,b(0,0)\ge\frac{S(Q)}{2\alpha}. \end{equation} Consequently, every minimizer $\gamma$ of $\cP_\kappa$ over $\cU_r$ satisfies \begin{equation}\label{I:hw:lambdatop} \lambda(0)\le\max\Big\{1,\frac{2\alpha C_\kappa}{s_\alpha}\Big\}\sqrt\delta . \end{equation} \end{lemma} \begin{proof} We first show that $S/\lambda^2$ is nondecreasing on $[0,r]$ and that $S<\alpha$, for every $\gamma\in\cU_r$. Let first $\gamma$ be a step order parameter. On each interval of constancy of $\gamma$, $\cL c=\gamma c^2$ gives $\cL(c^2)=2c\,\cL c+(\partial_xc)^2=2\gamma c^3+(\partial_xc)^2$, so It\^o's formula gives \begin{equation}\label{I:hw:c2} \frac{\dd}{\dd q}\E c(q,X_q)^2=\E\big[(\partial_xc)^2+2\gamma c^3\big](q,X_q)\ge0 \end{equation} there, because $c>0$. The function $q\mapsto\E c(q,X_q)^2$ is continuous, because $c$ is continuous and bounded by $(1-r)^{-1}$, so \eqref{I:hw:c2} shows that it is nondecreasing. For general $\gamma\in\cU_r$ take step order parameters $\gamma_n\in\cU_r$ with $\gamma_n\to\gamma$ in $L^1$. By \Cref{I:pf:coef-cont}, $c_{\gamma_n}\to c$ locally uniformly on $[0,r]\times\R$ and $X^{\gamma_n}_q\to X_q$ in $L^2$; since $00$, and we use \eqref{I:hw:stat} with $\cH$ as in the proof of \Cref{I:fun:value}. The support of $\mu$ contains $Q$ and a second point, which lies in $(0,Q)$ because $0\notin\supp\mu$. Every $p\in\supp\mu\cap(0,Q)$ is an interior minimum point of $\cH$ on $[0,r]$, so $D(p)=-2\cH'(p)=0$. By \Cref{I:fun:value}, $D(Q)\ge0$; this includes the case $Q=r$ of an active cutoff. One of two cases occurs. Suppose first that there are $p_n\in\supp\mu$ with $00$, so $p=\max(\supp\mu\cap[0,Q))$ exists, $00$ because every neighborhood of $p\in\supp\mu$ has positive mass while $\mu((p,Q))=0$. Then \begin{equation}\label{I:hw:gapint} 0\le D(Q)-D(p)=\int_p^Q\frac{S(q)-1}{\lambda(q)^2}\dd q . \end{equation} If $\alpha\le1$, the integrand is negative because $S<\alpha$, which is impossible; so $\alpha>1$. Put \[ \sigma=S(Q),\qquad \lambda_0=\lambda(Q)=\delta,\qquad \lambda_1=\lambda(p)=\lambda_0+\gamma(p)(Q-p)>\lambda_0 . \] On $[p,Q]$ the function $\lambda$ is affine with $\lambda'=-\gamma(p)$. We change variables to $\ell=\lambda(q)$, so that $\dd\ell=-\gamma(p)\dd q$, and bound $S(q)/\lambda(q)^2$ in two ways: by $S(Q)/\lambda(Q)^2=\sigma/\lambda_0^2$, since $S/\lambda^2$ is nondecreasing, and by $\alpha/\ell^2$, since $S<\alpha$. Multiplying \eqref{I:hw:gapint} by $\gamma(p)$ we obtain \begin{equation}\label{I:hw:gapineq} \frac1{\lambda_0}-\frac1{\lambda_1} \le\int_{\lambda_0}^{\lambda_1}\min\Big(\frac\sigma{\lambda_0^2},\frac\alpha{\ell^2}\Big)\dd\ell . \end{equation} If $\lambda_1\le2\lambda_0$, the left side of \eqref{I:hw:gapineq} is $(\lambda_1-\lambda_0)/(\lambda_0\lambda_1)\ge(\lambda_1-\lambda_0)/(2\lambda_0^2)$ and the right side is at most $\sigma(\lambda_1-\lambda_0)/\lambda_0^2$, so $\sigma\ge\frac12$. If $\lambda_1>2\lambda_0$, the left side is at least $1/(2\lambda_0)$, and the right side is at most the integral over $(0,\infty)$. Since $\sigma>0$ (because $c>0$), the two terms of the minimum agree at $\ell_*=\lambda_0\sqrt{\alpha/\sigma}$, so this integral equals $\sigma\ell_*/\lambda_0^2+\alpha/\ell_*=2\sqrt{\alpha\sigma}/\lambda_0$. Hence $\frac12\le2\sqrt{\alpha\sigma}$, that is, $\sigma\ge1/(16\alpha)$. Both cases give $S(Q)\ge s_\alpha$. For \eqref{I:hw:topslope}, let $\gamma\in\cU$. Since $\lambda(Q)=\delta$ and $c(Q,x)=V((x-\kappa)/\sqrt\delta)/\delta$, \begin{equation}\label{I:hw:SQ} S(Q)=\alpha\E V(Z)^2 . \end{equation} \Cref{I:hw:gauss}(iii) and \eqref{I:hw:mart} give $S(Q)\le2\alpha\E R(Z)=2\alpha\sqrt\delta\,b(0,0)$. Finally, let $\gamma$ minimize over $\cU_r$. If $\mu_\gamma$ has a single support point, then $\gamma=\1_{[Q,1]}$ and $\lambda(0)=\delta\le\sqrt\delta$. Otherwise the first assertion and \eqref{I:hw:topslope} give $\sqrt\delta\,b(0,0)\ge s_\alpha/(2\alpha)$, and \Cref{I:fun:slope} gives $\lambda(0)0$. Since $\gamma\ge0$ and $b>0$, $X_Q=B_Q+\int_0^Q\gamma b\dd q\ge B_Q$. Using \eqref{I:hw:SQ}, the bound $V<1$ on $\{Z0$ we have $\E(\kappa+1-\sqrt q\,G)_+^2=q\,m\big((\kappa+1)/\sqrt q\big)\le q\,m(\kappa+1)$, and this also holds for $q=0$. With $\alpha m(\kappa+1)\le\frac12$ and $\delta=1-q$ we obtain \[ \PRS_\kappa(q)\ge\frac{q}{4\delta}+\frac12\log\delta-\alpha C_0 =\frac14\Big(\frac1\delta-2\log\frac1\delta\Big)-\frac14-\alpha C_0\ge-\frac14-\alpha C_0 , \] because $y\ge2\log y$ for all $y>0$. \end{proof} \begin{proposition}\label{I:fun:finite} For every $\alpha>0$, \[ -\infty<\kc(\alpha)\le\kRS(\alpha)<\infty, \] where $\kRS(\alpha)$ is the unique solution of $\alpha\,m(\kRS)=1$. In particular $\kc(\alpha)<0$ if $\alpha>2$. \end{proposition} \begin{proof} The function $m$ is continuous and strictly increasing, since $m'(k)=2\E(k-G)_+>0$, and it tends to $0$ as $k\to-\infty$ and to $\infty$ as $k\to\infty$. Hence $\kRS$ exists, is unique, and is finite. \proofpart{The upper bound.} Let $\alpha m(\kappa)>1$. The upper bound in \eqref{I:hw:tail} gives \[ \PRS_\kappa(q)\le\frac{q-\alpha\E(\kappa-\sqrt q\,G)_+^2}{2(1-q)}+\frac12\log(1-q). \] As $q\uparrow1$ the numerator tends to $1-\alpha m(\kappa)<0$ by dominated convergence, so the right side tends to $-\infty$. Hence $\cP_*(\kappa)=-\infty$ whenever $\alpha m(\kappa)>1$, and $\kc\le\kRS$. \proofpart{The lower bound.} Let $\kappa_0$ be as in \Cref{I:hw:deep}. By that lemma and \eqref{I:hw:RSlower}, $\cP_*^r(\kappa_0)\ge\inf_{q\in[0,1)}\PRS_{\kappa_0}(q)\ge-\frac14-\alpha C_0$ for every $r<1$. Letting $r\uparrow1$ gives $\cP_*(\kappa_0)>-\infty$, hence $\kc\ge\kappa_0>-\infty$. Finally $m(0)=\E G_-^2=\frac12$, so $\alpha m(0)>1$ if $\alpha>2$. Since $m$ is increasing and $\alpha m(\kRS)=1$, this gives $\kc\le\kRS<0$. \end{proof} El Alaoui and Sellke observed the bound $\kc\le\kRS$, stated in terms of the constraint density, in the remark after \cite[Conjecture~1]{ElAlaouiSellke2022}; their argument is the one above, with $\gamma=\1_{[q,1]}$ and $q\to1$. For $\alpha>2$, combined with \Cref{I:thm:capacity}(ii), the bound $\kc\le\kRS$ recovers the replica-symmetric upper bound on the capacity, which Stojnic proved for every margin \cite[Section~2.3]{Stojnic2013Gardner}. \section{Divergence at the critical margin}\label{I:hw:critical-sec} Since $\kc$ is finite, we can approach it from both sides. Above $\kc$ the value is $-\infty$, but the constrained minima $\cP_*^r(k)$ of \eqref{I:hw:mr} take every sufficiently negative finite value. The value inequality \eqref{I:fun:valueineq} and the slope bound transport such a value down to $\kc-\eta$. \begin{theorem}\label{I:fun:critical} For every $\alpha>0$ the function $\cP_*$ is finite, nonincreasing, concave, and continuous on $(-\infty,\kc)$, and $\cP_*(\kappa)=-\infty$ for $\kappa\ge\kc$. More precisely, with $C_*=C_{\kc}$, for all sufficiently small $\eta>0$, \begin{equation}\label{I:hw:critbound} \cP_*(\kc-\eta)\le\frac12\log\eta+\frac12\log(2\alpha C_*)+\frac12 . \end{equation} \end{theorem} \begin{proof} Let $\kappa<\kc$. By the definition of $\kc$ as a supremum there is $\kappa'>\kappa$ with $\cP_*(\kappa')>-\infty$, and monotonicity (\Cref{I:fun:slope}) gives $\cP_*(\kappa)\ge\cP_*(\kappa')>-\infty$; also $\cP_*(\kappa)<0$ by \eqref{I:hw:test}. Thus $\cP_*$ is finite on the open interval $(-\infty,\kc)$, which is nonempty by \Cref{I:fun:finite}, and it is nonincreasing and concave there by \Cref{I:fun:slope}; a finite concave function on an open interval is continuous. For $\kappa>\kc$, $\cP_*(\kappa)=-\infty$ by the definition of $\kc$. Fix $k>\kc$. We first show that $r\mapsto\cP_*^r(k)$ is continuous on $[0,1)$. Let $r\e^{2w}$, and hence $\lambda_\gamma(0)\ge1-Q_\gamma>\e^{2w}$. For $k'0$ and $w<\alpha\log\Phi(-\kc)$. The function $k\mapsto\alpha\log\Phi(-k)$ is continuous, so $w<\alpha\log\Phi(-k)$ for all $k>\kc$ sufficiently close to $\kc$, and for such $k$ the bound holds with $k'=\kc-\eta$. Letting $k\downarrow\kc$ and using the continuity of $k\mapsto C_k$, we obtain \[ \cP_*(\kc-\eta)\le w+\alpha\eta C_*\e^{-2w}. \] The choice $w=\frac12\log(2\alpha C_*\eta)$ makes the last term equal to $\frac12$, and it is admissible when $\frac12\log(2\alpha C_*\eta)<\alpha\log\Phi(-\kc)$, that is, for all sufficiently small $\eta$. This proves \eqref{I:hw:critbound}. Since $\cP_*$ is nonincreasing, $\cP_*(\kc)\le\cP_*(\kc-\eta)$ for every such $\eta$, so $\cP_*(\kc)=-\infty$. \end{proof} We return to the deep margins of \Cref{I:hw:deep}. There the variational problem is replica symmetric: the infimum is a minimum over the one-parameter family \eqref{I:hw:RS}. \begin{corollary}\label{I:hw:rsdeep} Let $\kappa_0=\kappa_0(\alpha)$ be as in \Cref{I:hw:deep}, and let $\kappa\le\kappa_0$. Then $\kappa<\kc$, and every minimizer of $\cP_\kappa$ over $\cU$ equals $\1_{[q,1]}$ for some $q\in(0,1)$. In particular \[ \cP_*(\kappa)=\min_{q\in[0,1)}\PRS_\kappa(q)>-\infty . \] If $\alpha\le1$, the same holds for every $\kappa<\kc$. \end{corollary} \begin{proof} By \Cref{I:hw:deep} and the proof of \Cref{I:fun:finite}, $\cP_*(\kappa)\ge\cP_*(\kappa_0)>-\infty$, so minimizers exist by \Cref{I:fun:existence}, and $\kappa<\kc$ because $\cP_*(\kc)=-\infty$ by \Cref{I:fun:critical}. A minimizer $\gamma$ minimizes $\cP_\kappa$ over $\cU_r$ for every $r\in(Q,1)$, so $\gamma=\1_{[Q,1]}$ by \Cref{I:hw:deep}, and $Q>0$ by \eqref{I:hw:stat}. Hence $\cP_*(\kappa)=\PRS_\kappa(Q)\ge\inf_q\PRS_\kappa(q)\ge\cP_*(\kappa)$, and the infimum is attained at $q=Q$. If $\alpha\le1$ and $\kappa<\kc$, minimizers exist by \Cref{I:fun:existence}, and \Cref{I:fun:top} shows directly that each of them has a single support point. \end{proof} \section{Pinning of the top and collapse of minimizers}\label{I:hw:pin-sec} By \eqref{I:fun:valueineq}, a finite value keeps the top of a minimizer away from one. For a minimizer over all of $\cU$ the value identity is exact, and it also gives the converse: a very negative value forces the top close to one. The reason is that the terminal term $\alpha\E h(Z)$ is only logarithmically negative, since $h(z)\ge-\log(1+z_-)-C_0$, and the slope bound controls the negative part of $Z$. \begin{lemma}\label{I:fun:pin} Let $\kappa\in\R$ with $\cP_*(\kappa)>-\infty$, let $\gamma$ minimize $\cP_\kappa$ over $\cU$, and put $\delta=1-Q_\gamma$. Then \begin{equation}\label{I:hw:pin} \frac{1+\alpha}2\log\delta-\alpha\big(C_0+\log(1+C_\kappa)\big)\le\cP_*(\kappa)<\frac12\log\delta , \end{equation} with $C_0=-\log\Phi(-1)$ as in \Cref{I:hw:gauss} and $C_\kappa$ as in \Cref{I:fun:slope}. \end{lemma} \begin{proof} The upper bound in \eqref{I:hw:pin} is \eqref{I:fun:valueineq}. For the lower bound, the last assertion of \Cref{I:fun:value} gives $\cP_*(\kappa)=\cP_\kappa(\gamma)=\frac12\log\delta+\alpha\E h(Z)$. We first show that \[ h(z)\ge-\log(1+z_-)-C_0\qquad(z\in\R). \] For $z\ge0$ this holds because $h(z)\ge\log\Phi(z)\ge\log\frac12>-C_0$. For $z=-t<0$, \eqref{I:hw:tail} gives $\log\Phi(-t)\ge-\frac12t^2-\log(1+t)-C_0$, and $R(-t)>t$ by \Cref{I:hw:gauss}(i), so $\frac12R(z)^2\ge\frac12t^2$. Since $x\mapsto-\log(1+x)$ is convex on $[0,\infty)$, Jensen's inequality gives $\E h(Z)\ge-\log(1+\E Z_-)-C_0$. By \Cref{I:hw:gauss}(i), \eqref{I:hw:mart}, \Cref{I:fun:slope} and $\lambda(0)\ge\delta$, \[ \E Z_-\le\E R(Z)=\sqrt\delta\,b(0,0)<\frac{\sqrt\delta\,C_\kappa}{\lambda(0)}\le\frac{C_\kappa}{\sqrt\delta}. \] Since $\delta\le1$, $1+C_\kappa/\sqrt\delta\le(1+C_\kappa)/\sqrt\delta$. Hence \[ \cP_*(\kappa)\ge\frac12\log\delta-\alpha\log\frac{1+C_\kappa}{\sqrt\delta}-\alpha C_0 =\frac{1+\alpha}2\log\delta-\alpha\big(C_0+\log(1+C_\kappa)\big). \] \end{proof} Combined with the upper bound of \Cref{I:fun:critical} at the margin $\kc-\eta$, the lemma gives the collapse of minimizers used in \Cref{I:sec:gaps}. \begin{proposition}\label{I:fun:collapse-prop} Let $\alpha>0$. There are constants $C,\eta_0>0$, depending only on $\alpha$, such that for $0<\eta\le\eta_0$ every minimizer $\gamma$ of $\cP_{\kc-\eta}$ over $\cU$ satisfies \begin{equation}\label{I:hw:collapserates} 1-Q_\gamma\le C\eta^{1/(1+\alpha)},\qquad \lambda_\gamma(0)\le C\eta^{1/(2+2\alpha)} . \end{equation} \end{proposition} \begin{proof} Put $C_*=C_{\kc}$. Since $C_\kappa$ is increasing in $\kappa$, $C_\kappa\le C_*$ for $\kappa\le\kc$. By \Cref{I:fun:critical} there is $\eta_0\in(0,1]$ such that $\cP_*(\kc-\eta)\le\frac12\log\eta+C_1$ for $0<\eta\le\eta_0$, with $C_1=\frac12\log(2\alpha C_*)+\frac12$. Fix such $\eta$, put $k=\kc-\eta$, and let $\gamma$ be a minimizer of $\cP_k$ over $\cU$, which exists by \Cref{I:fun:existence}; write $\delta=1-Q_\gamma$. By \eqref{I:hw:pin} with $C_k\le C_*$, \[ \frac{1+\alpha}2\log\delta-C_2\le\cP_*(k)\le\frac12\log\eta+C_1,\qquad C_2=\alpha\big(C_0+\log(1+C_*)\big), \] so $\delta\le C_3\eta^{1/(1+\alpha)}$ with $C_3=\e^{2(C_1+C_2)/(1+\alpha)}$. The minimizer $\gamma$ minimizes $\cP_k$ over $\cU_r$ for every $r\in(Q_\gamma,1)$, so \eqref{I:hw:lambdatop} applies: with $C_4=\max\{1,2\alpha C_*/s_\alpha\}$, $\lambda_\gamma(0)\le C_4\sqrt\delta\le C_4C_3^{1/2}\eta^{1/(2+2\alpha)}$. \end{proof} The proof uses the exact value of $\cP_*$ at a minimizer, so it says nothing about the tops of near-minimizers. For $\alpha>2$, the bound $1-Q_\gamma\le C\eta^{1/(1+\alpha)}$ for every $\gamma$ with $\cP_{\kc-\eta}(\gamma)\le\cP_*(\kc-\eta)+1$, and a lower bound $\cP_*(\kc-\eta)\ge\alpha\log\eta-C$ of the same logarithmic order as \eqref{I:hw:critbound}, are proved in Part~II through a comparison with the zero-temperature functional (Part~II, \Cref{II:fun:rates}); in this part only \Cref{I:cor:logvolume} uses the lower bound. \Cref{I:fun:critical} and \Cref{I:fun:collapse-prop} are the bounds behind the comparison with the physical picture of jamming in \Cref{I:sec:intro}; the mass $\lambda_\gamma(0)$ measures the order parameter below the top. \section{Proof of the theorem on the variational problem}\label{I:hw:proof-jamming} We collect the results of this chapter; recall that $Q_\gamma=1-\delta$ is the top of the support of $\mu_\gamma$ and that $\lambda_\gamma(0)=\int_0^1\gamma$. \begin{proof}[Proof of \Cref{I:thm:jamming}] Part (i) is \Cref{I:fun:existence}, part (ii) is \Cref{I:fun:finite}, and part (iii) is \Cref{I:fun:critical}, with $C_*=C_{\kc}$. In part (iv), the pinning inequality is \Cref{I:fun:pin}, and the bounds on minimizers at the margin $\kc-\eta$ are \eqref{I:hw:collapserates} in \Cref{I:fun:collapse-prop}. Part (v) is \Cref{I:hw:rsdeep}. This proves the theorem. \end{proof} \chapter{The critical margin near \texorpdfstring{$\alpha=2$}{alpha=2}}\label{I:sec:gardner} This chapter proves \Cref{I:thm:gardner}: for constraint densities slightly above two, the critical margin lies below Gardner's prediction $\kRS$ by at least a constant times the cube of the distance to two. Throughout the chapter, $\alpha=2+e$ with $e>0$ small. Step~4 of \Cref{I:sec:ideas} outlines the argument. It uses only the definition \eqref{I:eq-kappac} of $\kc$, the Cole--Hopf recursion of \Cref{I:pf:step} for one explicit order parameter, the tail bound \eqref{I:hw:tail} and one-dimensional Gaussian calculus, part of which is taken from Part~II (\Cref{II:fun:gauss,II:lem:krs}). \Cref{I:cub:krs} records the behavior of $\kRS$ near $\alpha=2$. \Cref{I:cub:twolevel} bounds the value of an explicit order parameter with two support points near one by an explicit Gaussian integral $J_\ell$, and deduces that one negative value of $J_\ell$ bounds $\kc$ from above. \Cref{I:cub:expansion} expands $J_\ell$, and the proof of \Cref{I:thm:gardner} combines the three lemmas. In the notation of Part~II, $J_\ell(k)=J^k(\ell^{-1}\1_{[1-\ell,1)})$ is the value of the zero-temperature functional $J^k$ (Part~II, \eqref{II:fun:J}) at the coefficient $\ell^{-1}$ carried by the block $[1-\ell,1)$. Part~II proves the bound of \Cref{I:thm:gardner} with this coefficient (Part~II, \Cref{II:thm:cubic}), and transfers it to $\kc$ through the characterization of $\kc$ as the threshold of $J^k$ (Part~II, \Cref{II:thm:kJ}). The same characterization and a dual certificate for $J^k$ give the matching bound $\kRS-\kc\le Ce^3$, so the order $e^3$ is sharp (\Cref{I:cor:thresholds}). \Cref{I:cub:expansion} is a two-sided form of the bound in Step~3 of the proof of Part~II, \Cref{II:thm:cubic}. Here \Cref{I:cub:twolevel} bounds $\cP_\kappa$ by $J_\ell$ directly, so we need only the explicit form of $J_\ell$ and no comparison of $\cP_\kappa$ with $J^k$. Stojnic proved upper bounds on the capacity through a partially lifted version of his random duality theory, and numerical evaluation indicates that they lie below Gardner's prediction \cite{Stojnic2013}; he later proposed fully lifted versions, evaluated numerically at finite levels of lifting \cite{Stojnic2023}. The upper bound of Montanari, Zhong, and Zhou, discussed after \Cref{I:cor:gardner}, is expressed through the Gaussian exponential moment $\E\exp(-t(k-G)_+^2)$ \cite[Definition~2]{MontanariZhongZhou2024}, and $\e^{\cE(z)}$ in \eqref{I:cub:E} below is this moment at $t=\frac12$ and $k=-z$. The quadratic penalty that replaces the hard-wall datum in \Cref{I:cub:twolevel} is the jamming-limit boundary condition of Franz and Parisi \cite[(15)]{FranzParisi2016}, whose scaling regime near jamming also places the relevant overlaps close to one. What differs here is the trial order parameter, whose second atom sits at the bottom of a thin layer of relative width $\ell\asymp e^2$ below the top, and the analysis of the gain of this layer to order $e^3$ near $\alpha=2$, where $\kRS$ is close to zero. Besides $m(k)=\E(k-G)_+^2$ we use $m_1(k)=\E(k-G)_+$ and $\phi_0=\phi(0)=(2\pi)^{-1/2}$. By Part~II, \Cref{II:fun:gauss}(g), \begin{equation}\label{I:cub:mformulas} m(k)=(1+k^2)\Phi(k)+k\phi(k),\qquad m_1(k)=k\Phi(k)+\phi(k),\qquad m'=2m_1,\qquad m_1'=\Phi , \end{equation} and $\Phi-m=-k\,m_1$. In particular $m_1>0$, $m$ is increasing and convex, and $m_1$ is increasing. For $\alpha=2+e$, Gardner's prediction $\kRS$ is defined by $\alpha m(\kRS)=1$ (\Cref{I:fun:finite}). \begin{lemma}\label{I:cub:krs} There is a numerical constant $e_1\in(0,1]$ such that for $00 . \end{equation} \end{lemma} \begin{proof} Let $e_{\rm g}$ be the numerical constant of Part~II, \Cref{II:lem:krs}, and put $e_1=\min\{e_{\rm g},1\}$. For $00$ because $m_1>0$. \end{proof} Part~II, \Cref{II:lem:krs}(a), also gives the expansion of $\kRS$ quoted in the introduction: for $0k$. As $\delta\downarrow0$ we have $Q\to1$ and $\kappa'/\sqrt Q\to\kappa'>k$, so for all sufficiently small $\delta$ the hypothesis $\delta<(1-\delta)\ell$ holds and $\kappa'/\sqrt Q\ge k$. For such $\delta$, \eqref{I:cub:upper} and the monotonicity of $J_\ell$ give \[ \cP_*(\kappa')\le\cP_{\kappa'}(\gamma_\delta)\le\frac Q\delta J_\ell(k)+\frac12\log\delta , \] and the right side tends to $-\infty$ as $\delta\downarrow0$. Hence $\cP_*(\kappa')=-\infty$ for every $\kappa'>k$, and \eqref{I:eq-kappac} gives $\kc\le k$. \end{proof} The next lemma expands $J_\ell$ in the width $\ell$. The first term of the expansion is the value obtained from the replica-symmetric order parameters, and the second is the gain produced by the thin layer. \begin{lemma}\label{I:cub:expansion} There is a numerical constant $C_J$ such that for $0<\alpha\le3$, $0<\ell\le\frac12$, and $\abs k\le1$, \begin{equation}\label{I:cub:expand} \Big|J_\ell(k)-\frac{1-\alpha m(k)}4+\frac\ell2\,\mathsf d\,\big(\alpha\Phi(k)-1\big)\Big|\le C_J\ell^{3/2}, \qquad \mathsf d=\log2-\frac12 . \end{equation} Moreover $\mathsf d>0$. \end{lemma} \begin{proof} We divide the proof into three steps. First we isolate an integrable remainder of $\cE$, then we rewrite $J_\ell$ exactly with it, and finally we expand the explicit terms. \proofstep{1}{An integrable remainder.} Put \[ \cR(z)=\cE(z)+\frac14z_-^2+\frac{\log2}2\1_{\{z<0\}}\qquad(z\in\R). \] Let $z<0$ and write $A(z)=\e^{-z^2/4}/\sqrt2$, so that $\cR(z)=\log(\e^{\cE(z)}/A(z))$ by \eqref{I:cub:A}. Since $(z+G)_-^2\le(z+G)^2$, with equality on $\{z+G\le0\}$, \[ 0\le\e^{\cE(z)}-A(z)=\E\Big[1-\e^{-(z+G)^2/2};\,z+G>0\Big]\le\Prob(z+G>0)=\Phi(z). \] Hence $0\le\cR(z)\le\log(1+\Phi(z)/A(z))\le\sqrt2\,\e^{z^2/4}\Phi(z)$. For $-1\le z<0$ the right side is at most $\sqrt2\,\e^{1/4}$, and for $z=-t\le-1$ the Mills bound of \Cref{I:pf:gauss}(i) gives $\sqrt2\,\e^{t^2/4}\Phi(-t)\le\sqrt2\,\e^{t^2/4}\phi(t)/t\le\sqrt2\,\phi_0\e^{-t^2/4}$. Now let $z\ge0$. Then $\Phi(z)\le\e^{\cE(z)}\le1$, so $0\le-\cR(z)=-\cE(z)\le-\log\Phi(z)=-\log(1-\Phi(-z))\le2\Phi(-z)$, because $\Phi(-z)\le\frac12$ and $-\log(1-y)\le2y$ for $0\le y\le\frac12$. Since $\int_0^\infty\Phi(-z)\dd z=\E G_+=\phi_0$, these bounds give $\norm{\cR}_{L^1(\R)}<\infty$. \proofstep{2}{An exact formula.} Write $Y=Y_\ell(k)$ and, as in Part~II, put $m(k;t)=\E(k-\sqrt t\,G)_+^2$ for $t>0$, so that $m(k;1)=m(k)$. Then $\E Y_-^2=m(k;1-\ell)/\ell$ and $\Prob(Y<0)=\Phi(k/\sqrt{1-\ell})$, so the definitions \eqref{I:cub:J} of $J_\ell$ and of $\cR$ give \begin{equation}\label{I:cub:exact} J_\ell(k)=\frac{1-\ell-\alpha m(k;1-\ell)}4 +\frac{\ell\log2}2\Big(1-\alpha\Phi\big(k/\sqrt{1-\ell}\big)\Big)+\alpha\ell\,\E\cR(Y). \end{equation} The variable $Y$ is Gaussian with variance $(1-\ell)/\ell\ge1/(2\ell)$, so its density is at most $\phi_0\sqrt{2\ell}$, and \[ \abs{\alpha\ell\,\E\cR(Y)}\le3\sqrt2\,\phi_0\norm{\cR}_{L^1}\,\ell^{3/2}. \] \proofstep{3}{Expansion of the explicit terms.} Gaussian integration by parts, $\frac{\dd}{\dd t}\E F(\sqrt t\,G)=\frac12\E F''(\sqrt t\,G)$ applied to $F(y)=(k-y)_+^2$, gives $\partial_tm(k;t)=\Phi(k/\sqrt t)$. For $t\in[\frac12,1]$, \[ \Big|\frac{\dd}{\dd t}\Phi\big(k/\sqrt t\big)\Big|=\frac{\abs k\,\phi(k/\sqrt t)}{2t^{3/2}} \le\sqrt2\,\phi_0\abs k\le1 , \] so $\abs{\Phi(k/\sqrt t)-\Phi(k)}\le1-t$. Integrating over $t\in[1-\ell,1]$ gives $\abs{m(k;1-\ell)-m(k)+\ell\Phi(k)}\le\ell^2/2$, and also $\abs{\Phi(k/\sqrt{1-\ell})-\Phi(k)}\le\ell$. Inserting these bounds into \eqref{I:cub:exact}, \[ J_\ell(k)=\frac{1-\alpha m(k)}4+\frac\ell4\big(\alpha\Phi(k)-1\big) -\frac{\ell\log2}2\big(\alpha\Phi(k)-1\big)+\eps , \] where $\abs\eps\le\frac{\alpha\ell^2}8+\frac{\alpha\ell^2\log2}2+3\sqrt2\,\phi_0\norm{\cR}_{L^1}\ell^{3/2}$. The two middle terms combine to $-\frac\ell2\mathsf d(\alpha\Phi(k)-1)$, and $\ell^2\le\ell^{3/2}$, so \eqref{I:cub:expand} holds with $C_J=\frac38+\frac32\log2+3\sqrt2\,\phi_0\norm{\cR}_{L^1}$. Finally $\log2=\int_1^2t^{-1}\dd t>\frac12$, so $\mathsf d>0$. \end{proof} \begin{proof}[Proof of \Cref{I:thm:gardner}] Let $c_\Phi$, $C_J$, and $\mathsf d$ be as in \Cref{I:cub:krs,I:cub:expansion}, and put \[ t=\Big(\frac{\mathsf d\,c_\Phi}{4C_J}\Big)^2,\qquad c=\frac{t\,\mathsf d\,c_\Phi}8 . \] Let $e_0\le e_1$ be so small that $te_0^2\le\frac12$, $ce_0^3\le\frac12$, and $3t\mathsf de_0^2\le1$. Fix $00$ is small. If $\norm w^2\le tN$, then at most $tN/\tau^2$ rows have violation larger than $\tau$. The displacement must raise these rows by almost their violation, while it may lower every other row by up to $\tau$. A displacement with this property and with norm comparable to $\norm w$ exists when $A^{\mathsf T}$ is well conditioned on vectors supported on few rows. For $1\le m\le\min\{M,N\}$ we therefore define the smallest singular value of $A^{\mathsf T}$ over vectors supported on at most $m$ rows, \[ s_m(A)=\inf\bigl\{\norm{A^{\mathsf T}y}:\ y\in\R^M,\ \norm y=1,\ \abs{\supp y}\le m\bigr\}. \] \Cref{I:hv-matrix} shows that the Gaussian matrix $A$ has this property, together with a bound on its operator norm, with overwhelming probability, and \Cref{I:hv-sparse} then constructs the displacement. A lower bound on $s_m(A)$ is the lower half of the restricted isometry property \cite{CandesTao2005} for the $N\times M$ matrix $A^{\mathsf T}$, and we prove it by the net argument of \cite{BDDW2008}; see also \cite[Theorem~5.65]{Vershynin2012}. The proof of \Cref{I:hv-matrix} uses the operator-norm bound of \Cref{I:lo:opnorm}, nets, and one chi-square tail bound. For each $\eps\in(0,1]$, the unit sphere of $\R^n$ contains an $\eps$-net with at most $(1+2/\eps)^n$ points (volumetric bound). If $X$ has the chi-square law with $n$ degrees of freedom, then $\E\e^{-X/6}=(3/4)^{n/2}$, and Markov's inequality applied to $\e^{-X/6}$ gives \begin{equation}\label{I:vol:chi2} \Prob\{X\le3n/4\}\le\e^{n/8}\Bigl(\frac34\Bigr)^{n/2} =\e^{-n(\frac12\log\frac43-\frac18)}\le\e^{-n/64}, \end{equation} since $\frac12\log\frac43-\frac18>0.018>\frac1{64}$. \begin{lemma}\label{I:hv-matrix} There are constants $s\in(0,\min\{\alpha,1\}/4)$, $K\ge1$, and $c>0$, depending only on $\alpha$, such that for all sufficiently large $N$ the event \[ E_N=\Bigl\{\norm{A}_{\mathrm{op}}\le K,\quad s_{\lfloor2sN\rfloor}(A)\ge\tfrac12\Bigr\} \] has probability at least $1-\e^{-cN}$. \end{lemma} \begin{proof} We divide the proof into two steps: we first treat the operator norm and then the sparse singular values. \proofstep{1}{The operator norm.} Put $K=\sqrt{\alpha+1}+2$. Then $K\ge1$, and $K$ depends only on $\alpha$. \Cref{I:lo:opnorm}, applied to the matrix $\bG=\sqrt N A$ with $t=\sqrt N$, gives $\Prob\{\norm{A}_{\mathrm{op}}>\sqrt{M/N}+2\}\le2\e^{-N/2}$. Since $M\le(\alpha+1)N$, it follows that $\Prob\{\norm{A}_{\mathrm{op}}>K\}\le2\e^{-N/2}$. \proofstep{2}{Sparse singular values.} Let $s\in(0,\min\{\alpha,1\}/4)$, to be chosen below, and put $m=\lfloor2sN\rfloor$. For large $N$ we have $1\le m\le2sN\le\min\{M,N\}$, because $M\ge\alpha N/2$. Fix a set $S$ of $m$ rows, let $A_S$ be the $m\times N$ submatrix of $A$ formed by the rows in $S$, and fix a unit vector $v\in\R^S$. The vector $A_S^{\mathsf T}v=N^{-1/2}\sum_{a\in S}v_ag_a$ has the law $N(0,N^{-1}I_N)$, so $N\norm{A_S^{\mathsf T}v}^2$ has the chi-square law with $N$ degrees of freedom, and \eqref{I:vol:chi2} gives $\Prob\{\norm{A_S^{\mathsf T}v}^2\le3/4\}\le\e^{-N/64}$. Let $\mathcal N_S$ be a $1/(4K)$-net of the unit sphere of $\R^S$ with $\abs{\mathcal N_S}\le(1+8K)^m$. Suppose that $\norm{A}_{\mathrm{op}}\le K$ and that $\norm{A_S^{\mathsf T}v_0}^2>3/4$ for every $v_0\in\mathcal N_S$. For a unit vector $v\in\R^S$ choose $v_0\in\mathcal N_S$ with $\norm{v-v_0}\le1/(4K)$. Then \[ \norm{A_S^{\mathsf T}v}\ge\norm{A_S^{\mathsf T}v_0}-\norm{A}_{\mathrm{op}}\norm{v-v_0} \ge\frac{\sqrt3}2-\frac14>\frac12 . \] Every $y\in\R^M$ with $\abs{\supp y}\le m$ is supported in some set $S$ of exactly $m$ rows, and then $\norm{A^{\mathsf T}y}=\norm{A_S^{\mathsf T}y_S}$. Hence, if $\norm{A}_{\mathrm{op}}\le K$ and $s_m(A)<1/2$, then $\norm{A_S^{\mathsf T}v_0}^2\le3/4$ for some set $S$ of $m$ rows and some $v_0\in\mathcal N_S$. There are at most $\binom Mm\le(\e M/m)^m$ sets $S$, so a union bound over $S$ and over $v_0\in\mathcal N_S$ gives \[ \Prob\Bigl\{\norm{A}_{\mathrm{op}}\le K,\ s_m(A)<\tfrac12\Bigr\} \le\exp\Bigl\{m\log\frac{\e(1+8K)M}{m}-\frac N{64}\Bigr\}. \] The map $y\mapsto y\log(\e(1+8K)M/y)$ is increasing on $(0,(1+8K)M]$, and $m\le2sN\le M\le(\alpha+1)N$. Hence the exponent is at most $N\{2s\log(\e(1+8K)(\alpha+1)/(2s))-1/64\}$. Since $x\log(C/x)\to0$ as $x\downarrow0$, we may choose $s$, depending only on $\alpha$ and $K$, so small that $2s\log(\e(1+8K)(\alpha+1)/(2s))\le1/128$. The probability above is then at most $\e^{-N/128}$. Adding the two failure probabilities gives $\Prob(E_N^c)\le2\e^{-N/2}+\e^{-N/128}\le\e^{-N/256}$ for large $N$. This proves the lemma with $c=1/256$. \end{proof} The next lemma is an error bound of Hoffman type \cite{Hoffman1952} for the relaxed system $Ax\ge(k_0-2\tau)\1$: if the violation vector $(k_0\1-Au)_+$ has norm at most $\sqrt{tN}$, then the distance from $u$ to the solution set is at most $4\sqrt{tN}$. The constant $4$ does not depend on $N$, at the price of the slack $2\tau$ and of a smallness condition on $t$. \begin{lemma}\label{I:hv-sparse} Let $s$ and $K$ be as in \Cref{I:hv-matrix}, suppose that the event $E_N$ occurs, and let $N\ge1/s$. Let $k_0\in\R$, $\tau>0$, and $t>0$ with \[ t\le\frac{\tau^2s}{32K^2}. \] If $u\in\R^N$ satisfies $\norm{(k_0\1-Au)_+}^2\le tN$, then there is $d\in\R^N$ with \[ A(u+d)\ge(k_0-2\tau)\1,\qquad\norm d\le4\sqrt{tN}. \] \end{lemma} \begin{proof} Put $w=(k_0\1-Au)_+$ and $S=\{a:w_a>\tau\}$. By the definition of $w$ we have $Au\ge k_0\1-w$. It therefore suffices to find $d\in\R^N$ with \begin{equation}\label{I:eq:hv-goal} Ad\ge w-2\tau\1,\qquad\norm d\le4\norm w, \end{equation} because then $A(u+d)\ge(k_0-2\tau)\1$ and $\norm d\le4\sqrt{tN}$. The target $w-2\tau\1$ asks that each row be raised by its violation $w_a$ up to an error $2\tau$. For the rows $a\notin S$ this allows a decrease, since $w_a\le\tau$ there. We divide the proof into two steps. \proofstep{1}{Counting.} Since $\tau^2\abs S\le\norm w^2\le tN$ and $t\le\tau^2s/(32K^2)\le\tau^2s/32$, we have $\abs S\le sN/32$. Put $p=\lfloor sN\rfloor$ and $m=\lfloor2sN\rfloor$. As $sN\ge1$, we have $p\ge1$ and $p\ge sN/2$. The integer $\abs S+p$ is at most $(33/32)sN\le2sN$, hence at most $m$. Moreover $m\le\min\{M,N\}$, because $s<\min\{\alpha,1\}/4$ and $M\ge\alpha N/2$. On $E_N$, by the assumption on $t$, \begin{equation}\label{I:vol:zeta} s_m(A)-\frac{\norm{A}_{\mathrm{op}}\norm w}{\tau\sqrt p} \ge\frac12-\frac{K\sqrt{tN}}{\tau\sqrt{sN/2}} =\frac12-\frac{\sqrt2\,K\sqrt t}{\tau\sqrt s}\ge\frac14 . \end{equation} \proofstep{2}{Separation.} Put $R_0=4\norm w$ and $v=w-2\tau\1$. The required $d$ exists if and only if $v$ belongs to \[ \cK=\bigl\{Ad-z:\ d\in\R^N,\ \norm d\le R_0,\ z\in\R^M,\ z\ge0\bigr\}, \] since $v=Ad-z$ with $z\ge0$ means $Ad\ge v$. The set $\cK$ is convex. It is closed, being the sum of the compact set $\{Ad:\norm d\le R_0\}$ and the closed cone $\{-z:z\ge0\}$. Its support function $h_\cK(y)=\sup_{x\in\cK}y\cdot x$ equals $R_0\norm{A^{\mathsf T}y}$ if $y\ge0$: the supremum of $y\cdot Ad$ over $\norm d\le R_0$ is $R_0\norm{A^{\mathsf T}y}$, and $\sup_{z\ge0}(-y\cdot z)=0$. If some $y_a<0$, then $h_\cK(y)=+\infty$. By the separating hyperplane theorem, a point lies in the closed convex set $\cK$ if and only if $y\cdot v\le h_\cK(y)$ for every $y\in\R^M$. It therefore suffices to prove \begin{equation}\label{I:eq:hv-separation} y\cdot v\le R_0\norm{A^{\mathsf T}y}\qquad\text{for every }y\in\R^M\text{ with }y\ge0 . \end{equation} Fix $y\ge0$, and write $y_S$ and $y_{S^c}$ for its restrictions to $S$ and to $S^c$. For $a\in S$ we have $v_a\le w_a$, and for $a\notin S$ we have $v_a=w_a-2\tau\le-\tau$. Hence \[ y\cdot v\le y_S\cdot w_S-\tau\norm{y_{S^c}}_1 . \] The right side of \eqref{I:eq:hv-separation} is nonnegative, so we may assume $y\cdot v>0$. Then \begin{equation}\label{I:vol:sep-aux} \tau\norm{y_{S^c}}_10$, let $\xi$ be a standard Gaussian vector in $\R^N$, and put \[ Y(\sigma,\xi)=\sqrt N\,\frac{\sigma+\rho P_\sigma\xi}{\norm{\sigma+\rho P_\sigma\xi}}\in S_N . \] Since $P_\sigma\xi\perp\sigma$, we have $\norm{\sigma+\rho P_\sigma\xi}^2=N+\rho^2\norm{P_\sigma\xi}^2\ge N$. Thus $Y$ is a continuous map from $S_N\times\R^N$ to $S_N$, and $Y(\sigma,\xi)$ is obtained from $\sigma+\rho P_\sigma\xi$ by multiplication with a number in $(0,1]$. In particular, normalization can only raise a negative margin. Two properties of $Y$ are used. If $\sigma$ is uniform on $S_N$ and independent of $\xi$, then so is $Y(\sigma,\xi)$. For each fixed $\sigma$ that admits a small repair, the point $Y(\sigma,\xi)$ is feasible at a slightly smaller margin with probability at least $\e^{-N\eps}$ for a small $\eps>0$. The second property rests on two Gaussian inequalities. The first compares a symmetric event with its translate. It is a standard lower bound for the Gaussian measure of a shifted symmetric set; see \cite{KuelbsLiLinde1994,LiShao2001}. \begin{lemma}\label{I:vol:shift} Let $\xi$ be a standard Gaussian vector in $\R^N$, let $\cR\subseteq\R^N$ be a Borel set with $\cR=-\cR$, and let $y\in\R^N$. Then \[ \Prob\{\xi\in y+\cR\}\ge\e^{-\norm y^2/2}\,\Prob\{\xi\in\cR\}. \] \end{lemma} \begin{proof} The standard Gaussian density $\varphi_N$ on $\R^N$ satisfies $\varphi_N(x+y)=\varphi_N(x)\e^{-x\cdot y-\norm y^2/2}$. Hence \begin{align*} \Prob\{\xi\in y+\cR\}&=\int_\cR\varphi_N(x+y)\dd x =\e^{-\norm y^2/2}\,\E\bigl[\e^{-\xi\cdot y}\1_\cR(\xi)\bigr]\\ &=\e^{-\norm y^2/2}\,\E\bigl[\cosh(\xi\cdot y)\1_\cR(\xi)\bigr], \end{align*} where the last equality holds because $\xi$ and $-\xi$ have the same law and $\cR=-\cR$. Since $\cosh\ge1$, this proves the lemma. \end{proof} The second is \v{S}id\'ak's inequality in the form of \Cref{I:up-sidak}. In the next lemma the matrix $A$ is fixed, and probabilities refer to $\xi$ and to the auxiliary uniform point in (b). \begin{lemma}\label{I:hv-smoothing} Let $K>0$ be such that $\max_a\norm{b_a}\le K$. Let $k<0$, $00$, and $\rho>0$, suppose that $\exp(-\eta^2/(2K^2\rho^2))\le1/2$, and put \[ k'=\frac k{1-r}-\eta . \] \begin{enumerate}[label=\textup{(\alph*)}] \item If $\sigma\in S_N$ and $d\in\R^N$ satisfy $\norm d\le r\sqrt N$ and $A(\sigma+d)\ge k\1$, then \begin{equation}\label{I:vol:transfer-point} \Prob\bigl\{Y(\sigma,\xi)\in S_N(k')\bigr\} \ge\exp\Bigl\{-\frac{2r^2N}{\rho^2}-2M\e^{-\eta^2/(2K^2\rho^2)}\Bigr\}. \end{equation} \item If $B\subseteq S_N$ is a Borel set such that every $\sigma\in B$ admits a vector $d$ as in \textup{(a)}, then \begin{equation}\label{I:vol:transfer-set} V_N(k')\ge\mu_N(B)\exp\Bigl\{-\frac{2r^2N}{\rho^2}-2M\e^{-\eta^2/(2K^2\rho^2)}\Bigr\}. \end{equation} \end{enumerate} \end{lemma} \begin{proof} We divide the proof into four steps. The first three prove (a): we move the repair displacement into the tangent space, compare the perturbed point with its translate, and bound the probability of the untranslated event. The fourth proves (b) by integration. \proofstep{1}{A tangent displacement.} The Gaussian vector $\rho P_\sigma\xi$ lies in $\sigma^\perp$, so we first replace $d$ by a vector in $\sigma^\perp$. Write $d=c\sigma+P_\sigma d$ with $c=d\cdot\sigma/N$. Then $\abs c\le\norm d/\sqrt N\le r\le1/2$, and $d'=P_\sigma d/(1+c)\in\sigma^\perp$ satisfies \[ \sigma+d'=\frac{(1+c)\sigma+d-c\sigma}{1+c}=\frac{\sigma+d}{1+c}. \] Since $k<0$ and $0<1/(1+c)\le1/(1-r)$, \begin{equation}\label{I:vol:tangent} A(\sigma+d')=\frac{A(\sigma+d)}{1+c}\ge\frac k{1+c}\,\1\ge\frac k{1-r}\,\1, \qquad \norm{d'}\le\frac{\norm d}{1-r}\le2r\sqrt N . \end{equation} \proofstep{2}{Shifting the Gaussian vector.} Let \[ \cR=\bigl\{x\in\R^N:\abs{b_a\cdot P_\sigma x}\le\eta/\rho\text{ for all }a\le M\bigr\}, \] a closed set with $\cR=-\cR$, and put $y=d'/\rho$. If $\xi\in y+\cR$, then, since $P_\sigma y=y$, \[ \sigma+\rho P_\sigma\xi=\sigma+d'+\rho P_\sigma(\xi-y),\qquad A(\sigma+\rho P_\sigma\xi)\ge\frac k{1-r}\,\1-\eta\1=k'\1, \] by \eqref{I:vol:tangent}. Since $k'<0$ and $Y(\sigma,\xi)$ is obtained from $\sigma+\rho P_\sigma\xi$ by multiplication with a number in $(0,1]$, also $AY(\sigma,\xi)\ge k'\1$, that is, $Y(\sigma,\xi)\in S_N(k')$. By \Cref{I:vol:shift} and $\norm y^2=\norm{d'}^2/\rho^2\le4r^2N/\rho^2$, \begin{equation}\label{I:eq:hv-shift} \Prob\bigl\{Y(\sigma,\xi)\in S_N(k')\bigr\}\ge\Prob\{\xi\in y+\cR\} \ge\e^{-2r^2N/\rho^2}\,\Prob\{\xi\in\cR\}. \end{equation} \proofstep{3}{The untranslated event.} The variables $\zeta_a=b_a\cdot P_\sigma\xi=(P_\sigma b_a)\cdot\xi$, $a\le M$, are centered and jointly Gaussian, with $\Var\zeta_a=\norm{P_\sigma b_a}^2\le K^2$, and $\{\xi\in\cR\}=\{\abs{\zeta_a}\le\eta/\rho\text{ for all }a\}$. By \Cref{I:up-sidak}, the probability of this event is at least $\prod_a\Prob\{\abs{\zeta_a}\le\eta/\rho\}$. For $z\ge0$ the Gaussian tail bound $2\Prob\{G>z\}\le\e^{-z^2/2}$, together with the fact that $\Prob\{\abs{N(0,v)}>z\}$ is nondecreasing in $v$, gives $\Prob\{\abs{\zeta_a}>\eta/\rho\}\le\e^{-\eta^2/(2K^2\rho^2)}\le1/2$. Using $\log(1-x)\ge-2x$ for $0\le x\le1/2$, we obtain \[ \Prob\{\xi\in\cR\}\ge\bigl(1-\e^{-\eta^2/(2K^2\rho^2)}\bigr)^M \ge\exp\bigl\{-2M\e^{-\eta^2/(2K^2\rho^2)}\bigr\}. \] Together with \eqref{I:eq:hv-shift} this proves \eqref{I:vol:transfer-point}. \proofstep{4}{Integration.} Let $\Sigma$ be a random point with law $\mu_N$, independent of $\xi$. We first show that $Y(\Sigma,\xi)$ also has law $\mu_N$. For an orthogonal matrix $O$ we have $P_{O\sigma}=OP_\sigma O^{\mathsf T}$, and hence $Y(O\sigma,O\xi)=OY(\sigma,\xi)$. The pair $(O\Sigma,O\xi)$ has the same law as $(\Sigma,\xi)$, so $OY(\Sigma,\xi)=Y(O\Sigma,O\xi)$ has the same law as $Y(\Sigma,\xi)$. The law $\nu$ of $Y(\Sigma,\xi)$ is therefore a Borel probability measure on $S_N$ that is invariant under all orthogonal transformations, and such a measure equals $\mu_N$. Indeed, let $H$ be the Haar probability measure on the compact group $O(N)$, which is invariant under multiplication from the left and from the right. Let $f$ be a bounded Borel function on $S_N$. For $\sigma,\sigma'\in S_N$ choose $O_0\in O(N)$ with $\sigma'=O_0\sigma$. The right invariance of $H$ gives $\int f(O\sigma')H(\dd O)=\int f(OO_0\sigma)H(\dd O) =\int f(O\sigma)H(\dd O)$, so this integral is a constant $c_f$ that does not depend on $\sigma$. By the invariance of $\nu$ and Fubini's theorem, $\int f\dd\nu=\int\!\int f(O\sigma)H(\dd O)\,\nu(\dd\sigma)=c_f$, and the same computation gives $\int f\dd\mu_N=c_f$. Hence $\nu=\mu_N$. The map $Y$ is continuous and $S_N(k')$ is closed, so by independence and Fubini's theorem $\sigma\mapsto\Prob\{Y(\sigma,\xi)\in S_N(k')\}$ is Borel and \begin{align*} V_N(k')=\Prob\bigl\{Y(\Sigma,\xi)\in S_N(k')\bigr\} &\ge\Prob\bigl\{\Sigma\in B,\ Y(\Sigma,\xi)\in S_N(k')\bigr\}\\ &=\int_B\Prob\bigl\{Y(\sigma,\xi)\in S_N(k')\bigr\}\,\mu_N(\dd\sigma). \end{align*} By (a), the integrand is at least the right side of \eqref{I:vol:transfer-point} for every $\sigma\in B$. This proves \eqref{I:vol:transfer-set}. \end{proof} No measurable choice of the repair displacement is needed in (b): the lower bound \eqref{I:vol:transfer-point} holds at every point of $B$. \section{The hard-volume lower bound}\label{I:vol:lower} We now define the penalties to which the Parisi formula is applied. Fix $\chi\in C^\infty(\R)$ with values in $[0,1]$, $\chi=0$ on $(-\infty,0]$ and $\chi=1$ on $[1,\infty)$. For $\omega>0$, $\beta>0$, and $\ell\in\R$ put \[ h_\omega(y)=\int_0^y\int_0^{y'}2\chi(v/\omega)\dd v\dd y',\qquad U_{\beta,\ell}(x)=-\beta h_\omega(\ell-x). \] The dependence of $U_{\beta,\ell}$ on $\omega$ and $\chi$ is suppressed. Since $h_\omega''(y)=2\chi(y/\omega)$ vanishes for $y\le0$, lies in $[0,2]$ and equals $2$ for $y\ge\omega$, and since $h_\omega(0)=h_\omega'(0)=0$, the function $h_\omega$ vanishes on $(-\infty,0]$ and is nondecreasing. Moreover $h_\omega'(y)\ge2(y-\omega)$ for $y\ge\omega$, so that $h_\omega(y)\ge(y-\omega)_+^2$ for all $y$. Consequently: \begin{enumerate}[label=(P\arabic*)] \item\label{I:vol:P1} $U_{\beta,\ell}=0$ on $[\ell,\infty)$ and $U_{\beta,\ell}(x)\le-\beta(\ell-\omega-x)_+^2$ for $x\in\R$. In particular $H_\ell\le U_{\beta,\ell}\le0$, where $H_\ell(x)=\log\1_{\{x\ge\ell\}}$. \item\label{I:vol:P2} $U_{\beta,\ell}$ is smooth and concave, with $\sup U_{\beta,\ell}=0$ and $-2\beta\le U_{\beta,\ell}''\le0$. Thus $U_{\beta,\ell}$ belongs to class~\ref{I:class-B}. \end{enumerate} The following theorem gives the lower bounds in \Cref{I:thm:capacity}(i),(ii). The restriction to negative margins enters only through the transfer of \Cref{I:hv-smoothing}. \begin{theorem} \label{I:hv-hard-lower} Let $\alpha>0$ and $\kappa<\min\{\kc(\alpha),0\}$. Then $\cP_*(\kappa)$ is finite, and for every $\eps>0$, \begin{equation}\label{I:vol:lower-bound} \Prob\Bigl\{\frac1N\log V_N(\kappa)\ge\cP_*(\kappa)-\eps\Bigr\}\longrightarrow1. \end{equation} Consequently $\Prob\{\kappa_N\ge\kappa\}\to1$. \end{theorem} \begin{proof} We may assume $\eps\le1$. We divide the proof into five steps: regularity of $\cP_*$, a lower bound for the free energy of one penalty, the choice of parameters, the passage from the free energy to the volume of approximately feasible configurations, and the repair and transfer of \Cref{I:hv-sparse,I:hv-smoothing}. \proofstep{1}{Regularity of $\cP_*$.} By \Cref{I:fun:critical}, the function $\cP_*$ is finite, nonincreasing, and continuous on $(-\infty,\kc)$. In particular $\cP_*(\kappa)$ is finite and $\cP_*$ is continuous at $\kappa$. \proofstep{2}{The penalty free energy.} Let $\ell\in\R$ and $\beta,\omega>0$, and write $U=U_{\beta,\ell}$. We show that for every $\eps'>0$, \begin{equation}\label{I:vol:soft-prob} \Prob\Bigl\{\frac1N\log Z_N(U)\ge\cP_*(\ell)-\eps'\Bigr\}\longrightarrow1 . \end{equation} By \ref{I:vol:P2} and \Cref{I:thm:parisi}, $N^{-1}\log Z_N(U)=f_N(U)\to\cP_*(U)$ in probability, so it suffices to prove $\cP_*(U)\ge\cP_*(\ell)$. Let $\gamma\in\cU$ and $r\in[Q_\gamma,1)$. First let $\gamma\in\cU_r$ be a step order parameter. By \ref{I:vol:P1}, $H_\ell\le U$, so \eqref{I:pf:monotone} and \eqref{I:pf:Hkappa} give $u_\gamma(0,0;U)\ge u_\gamma(0,0;H_\ell)=u_\gamma(0,0;\ell)$. The entropy term $\CS(\gamma)$ does not depend on the activation, so $\cP_U(\gamma)\ge\cP_\ell(\gamma)\ge\cP_*(\ell)$. For general $\gamma\in\cU$, choose step order parameters $\gamma_n\in\cU_r$ with $\gamma_n\to\gamma$ in $L^1([0,1])$ (\Cref{I:pf:step}). Then $\cP_U(\gamma_n)\ge\cP_*(\ell)$ for every $n$. Since $U$ belongs to class~\ref{I:class-B}, \eqref{I:lo-phi-lip} in \Cref{I:lo-phi-cont} gives $\cP_U(\gamma_n)\to\cP_U(\gamma)$. Hence $\cP_U(\gamma)\ge\cP_*(\ell)$ for every $\gamma\in\cU$, that is, $\cP_*(U)\ge\cP_*(\ell)$, and \eqref{I:vol:soft-prob} follows. \proofstep{3}{Choice of parameters.} Let $s$ and $K$ be the constants of \Cref{I:hv-matrix}. They depend only on $\alpha$. We choose the remaining parameters in the following order, and each choice depends only on the earlier ones and on $\alpha,\kappa,\eps$. \begin{enumerate}[label=\textup{(\roman*)}] \item The level $\ell$. By Step~1, choose $\ell\in(\kappa,\min\{\kc,0\})$ with $\cP_*(\ell)\ge\cP_*(\kappa)-\eps/4$. Then $\cP_*(\ell)$ is finite. Put $\vartheta=(\ell-\kappa)/5$. The margin decreases from $\ell$ to $\kappa$ in four stages: the penalty has transition width $\omega=\vartheta$, the repair of \Cref{I:hv-sparse} with $\tau=\vartheta$ loses $2\vartheta$, and the transfer of \Cref{I:hv-smoothing} with $\eta=\vartheta$ loses at most $\vartheta$ through the factor $1/(1-r)$ and $\vartheta$ through $\eta$. \item The perturbation size $\rho>0$, chosen so that $2(\alpha+1)\exp(-\vartheta^2/(2K^2\rho^2))\le\eps/8$. Since $M\le(\alpha+1)N$ and $\eps\le1$, this gives $2M\exp(-\vartheta^2/(2K^2\rho^2))\le N\eps/8$ and $\exp(-\vartheta^2/(2K^2\rho^2))\le1/16$. \item The violation level $t>0$, chosen so small that \[ t\le\frac{\vartheta^2s}{32K^2},\qquad\frac{32t}{\rho^2}\le\frac\eps8, \qquad r:=4\sqrt t\le\min\Bigl\{\frac12,\frac\vartheta{2\abs\kappa}\Bigr\}. \] The first condition is the hypothesis of \Cref{I:hv-sparse} with $\tau=\vartheta$. The second bounds the shift cost $2r^2/\rho^2=32t/\rho^2$ in \Cref{I:hv-smoothing}. The third is the condition $r\le1/2$ of that lemma and bounds the margin lost in the factor $1/(1-r)$. \item The penalty strength $\beta=(\abs{\cP_*(\ell)}+1)/t$, so that configurations with total squared violation above $tN$ carry weight at most $\e^{-N(\abs{\cP_*(\ell)}+1)}$ in $Z_N(U)$. \end{enumerate} Write $U=U_{\beta,\ell}$ with $\omega=\vartheta$, and put $k_0=\ell-\vartheta$ and $k=\ell-3\vartheta<0$. \proofstep{4}{Approximately feasible configurations.} Let \[ B=\bigl\{\sigma\in S_N:\norm{(k_0\1-A\sigma)_+}^2\le tN\bigr\}, \] a closed subset of $S_N$. By \ref{I:vol:P1}, $\exp\{\sum_aU(b_a\cdot\sigma)\}\le\exp\{-\beta\norm{(k_0\1-A\sigma)_+}^2\}$, which is at most $1$ on $B$ and at most $\e^{-\beta tN}$ off $B$. Hence \[ Z_N(U)\le\mu_N(B)+\e^{-\beta tN}\le\mu_N(B)+\e^{N(\cP_*(\ell)-1)}, \] because $-\beta t=-\abs{\cP_*(\ell)}-1\le\cP_*(\ell)-1$. On the event in \eqref{I:vol:soft-prob} with $\eps'=\eps/8$ we have $Z_N(U)\ge\e^{N(\cP_*(\ell)-\eps/8)}$. Since $\cP_*(\ell)-1\le\cP_*(\ell)-\eps/8-7/8$, \[ \mu_N(B)\ge\e^{N(\cP_*(\ell)-\eps/8)}\bigl(1-\e^{-7N/8}\bigr) \ge\e^{N(\cP_*(\ell)-\eps/4)} \] as soon as $N\ge8\log2/\eps$, since then $1-\e^{-7N/8}\ge1/2\ge\e^{-N\eps/8}$. \proofstep{5}{Repair and transfer.} By \Cref{I:hv-matrix} and Step~4, with probability tending to one the event $E_N$ occurs and $\mu_N(B)\ge\e^{N(\cP_*(\ell)-\eps/4)}$. We work on this event, with $N\ge\max\{1/s,8\log2/\eps\}$. Let $\sigma\in B$. \Cref{I:hv-sparse}, applied with $u=\sigma$, $k_0=\ell-\vartheta$, $\tau=\vartheta$, and $t$ (condition (iii)), gives $d\in\R^N$ with $A(\sigma+d)\ge(k_0-2\vartheta)\1=k\1$ and $\norm d\le4\sqrt{tN}=r\sqrt N$. On $E_N$ we have $\max_a\norm{b_a}\le\norm{A}_{\mathrm{op}}\le K$, because $b_a=A^{\mathsf T}e_a$ for the $a$-th unit vector $e_a$ of $\R^M$. Condition (ii) gives $\exp(-\vartheta^2/(2K^2\rho^2))\le1/2$, so \Cref{I:hv-smoothing}(b) applies with this $K$, the margin $k<0$, $r\le1/2$, $\eta=\vartheta$, and $\rho$. By conditions (ii) and (iii), it gives \[ V_N(k')\ge\mu_N(B)\exp\Bigl\{-\frac{32tN}{\rho^2}-2M\e^{-\vartheta^2/(2K^2\rho^2)}\Bigr\} \ge\mu_N(B)\e^{-N\eps/4}, \qquad k'=\frac k{1-r}-\vartheta . \] The margin $k'$ is at least $\kappa$. Indeed, since $\kappa0$, because $\cP_*(\kappa)$ is finite. Then $S_N(\kappa)$ is nonempty, that is, some $\sigma\in S_N$ has $\min_ab_a\cdot\sigma\ge\kappa$, and $\kappa_N\ge\kappa$. Hence $\Prob\{\kappa_N\ge\kappa\}\to1$. \end{proof} \section{Proof of the capacity theorem}\label{I:vol:proof} \begin{proof}[Proof of \Cref{I:thm:capacity}] Let $\alpha>2$. By \Cref{I:fun:finite}, $-\infty<\kc(\alpha)\le\kRS(\alpha)<\infty$ and, since $\alpha>2$, $\kc(\alpha)<0$. \proofpart{Proof of (i).} Fix $\kappa<\kc$ and $\eps>0$. Since $\kappa<\kc<0$, \Cref{I:hv-hard-lower} shows that $\cP_*(\kappa)$ is finite and that \[ \Prob\Bigl\{\frac1N\log V_N(\kappa)\ge\cP_*(\kappa)-\eps\Bigr\}\longrightarrow1 . \] For the upper bound choose $\gamma\in\cU$ with $\cP_\kappa(\gamma)\le\cP_*(\kappa)+\eps/2$. \Cref{I:up-hardvolume}(i), applied with this $\gamma$, the margin $\kappa$, and $\eps/2$, gives $\Prob\{N^{-1}\log V_N(\kappa)>\cP_*(\kappa)+\eps\}\to0$. The two bounds prove (i). \proofpart{Proof of (ii).} Let $\eps>0$. By \Cref{I:hv-hard-lower} at the margin $\kc-\eps<0$, $\Prob\{\kappa_N<\kc-\eps\}\to0$. By \Cref{I:up-margin}, $\Prob\{\kappa_N\ge\kc+\eps\}\to0$. Hence $\kappa_N\to\kc$ in probability. \proofpart{Proof of (iii).} By \Cref{I:fun:critical}, $\cP_*(\kappa)=-\infty$ for every $\kappa\ge\kc$. Let $\kappa\ge\kc$ and $C<\infty$. Choose $\gamma\in\cU$ with $\cP_\kappa(\gamma)<-C-1$, and apply \Cref{I:up-hardvolume}(i) with this $\gamma$, the margin $\kappa$, and $\eps=1$: \[ \Prob\Bigl\{\frac1N\log V_N(\kappa)>-C\Bigr\}\le \Prob\Bigl\{\frac1N\log V_N(\kappa)>\cP_\kappa(\gamma)+1\Bigr\}\longrightarrow0 . \] This includes the critical margin $\kappa=\kc$. \end{proof} The assumption $\alpha>2$ enters the proof of part (i) only through the inequality $\kc(\alpha)<0$. \Cref{I:hv-hard-lower} and \Cref{I:up-hardvolume} hold for every $\alpha>0$, so the proof of part (i) shows that $N^{-1}\log V_N(\kappa)\to\cP_*(\kappa)$ in probability for every $\alpha>0$ and every $\kappa<\min\{\kc(\alpha),0\}$. Such $\kappa$ exist because $\kc(\alpha)>-\infty$ by \Cref{I:fun:finite}. \chapter{Gap statistics}\label{I:sec:gaps} This chapter proves \Cref{I:thm:gibbs-gaps,I:thm:gaps}; Step~6 of \Cref{I:sec:ideas} outlines the argument. Both theorems concern averages of a test function $\psi$ over the gaps $b_a\cdot\sigma-k$ of the constraints, and both rest on the deviation inequality of \Cref{I:gap:master}. It compares such averages over every set $T\subseteq S_N(k)$ of positive volume with $\alpha\E\psi(X^{\gamma,k}_1-k)$, up to an error that involves a source strength $\beta>0$, the volume term $\cP_k(\gamma)-N^{-1}\log\mu_N(T)$, and the mass $\lambda_\gamma(0)$. \Cref{I:gap:contacts-sec} counts the contacts at optimal configurations. \Cref{I:gap:recursion} writes the Parisi equation for a step order parameter as a finite recursion, introduces the Markov chain that describes the linearization of this recursion, and identifies that chain with the Parisi diffusion. \Cref{I:gap:transport} proves the quadratic source estimate, and \Cref{I:gap:gibbs} combines it with Jensen's inequality and the upper bound of \Cref{I:sec:upper} to prove \Cref{I:gap:master}. For \Cref{I:thm:gibbs-gaps}, at a fixed margin $\kappa<\kc$, $T$ consists of the configurations in $S_N(\kappa)$ whose gap averages exceed, or fall below, $\alpha\E\psi(X^{\gamma,\kappa}_1-\kappa)$ by more than $\eps$, for a minimizer $\gamma$ of $\cP_\kappa$ (\Cref{I:gap:proof-gibbs}). For \Cref{I:thm:gaps}, $T$ is the set of \Cref{I:up-cap} near one configuration $\sigma_0$ of nearly maximal margin (\Cref{I:gap:cap}), and \Cref{I:gap:proof-gaps} combines this with the collapse of minimizers (\Cref{I:fun:collapse-prop}) and with the contacts of \Cref{I:gap:contacts}. \paragraph{Notation.} Throughout this chapter, $\alpha_N=M/N\to\alpha$ and $\bar\alpha=\sup_N\alpha_N<\infty$, and $A=N^{-1/2}\bG$ is the matrix with rows $b_a$, as in \Cref{I:sec:hard-volume}. The letter $k$ denotes a real margin. A \emph{test function} is a bounded Lipschitz function $\psi:[0,\infty)\to\R$; we extend it to $\R$ by $\psi(x)=\psi(0)$ for $x<0$, which preserves its bound and its Lipschitz constant. We call $\psi$ $L$-Lipschitz if $\abs{\psi(x)-\psi(y)}\le L\abs{x-y}$ for all $x,y$. For $\sigma\in S_N$ and $k,\beta\in\R$ put \[ \Psi_N(\sigma,k)=\frac1N\sum_{a=1}^M\psi(b_a\cdot\sigma-k), \qquad f_N(k,\beta)=\frac1N\log\int_{S_N(k)}\e^{N\beta\Psi_N(\sigma,k)}\,\mu_N(\dd\sigma), \] so that $f_N(k,0)=N^{-1}\log V_N(k)$, and $f_N(k,\beta)=f_N(H_k+\beta\psi(\cdot-k))$ in the notation of \eqref{I:eq-model}. The test function is suppressed from the notation. For a Borel set $T\subseteq S_N$ with $\mu_N(T)>0$, $\langle\cdot\rangle_T$ denotes expectation under the probability measure $\mu_N(\cdot\cap T)/\mu_N(T)$, and $\langle\cdot\rangle_{N,k}=\langle\cdot\rangle_{S_N(k)}$ when $V_N(k)>0$. Thus $\int\psi\dd\nu_{N,\sigma}=\Psi_N(\sigma,m_N(\sigma))$, and the average in \Cref{I:thm:gibbs-gaps} is $\langle\Psi_N(\sigma,\kappa)\rangle_{N,\kappa}$. Several statements below involve a supremum over a random set of configurations, such as the set of maximizers of $m_N$. A statement that such an event has probability tending to one, has probability at least $1-\e^{-cN}$, or occurs almost surely means that it contains an event, defined without reference to the random set, with the same property. In particular no measurable selection of maximizers is needed. Let $C_{\mathrm{op}}=K$ be the constant of \Cref{I:hv-matrix}, which depends only on $\alpha$. By that lemma, $\norm{A}_{\mathrm{op}}\le C_{\mathrm{op}}$ with probability tending to one. Each $N\norm{b_a}^2$ has the chi-square law with $N$ degrees of freedom, and Markov's inequality applied to $\e^{N\norm{b_a}^2/4}$, whose mean is $2^{N/2}$, gives $\Prob\{N\norm{b_a}^2\ge2N\}\le\e^{-N/2}2^{N/2}=\e^{-N(1-\log2)/2}$. A union bound over $a\le M$ gives $\Prob\{\max_a\norm{b_a}^2>2\}\le\bar\alpha N\e^{-N(1-\log2)/2}\to0$. Hence the event \[ \Omega_N=\Bigl\{\norm{A}_{\mathrm{op}}\le C_{\mathrm{op}},\ \max_{a\le M}\norm{b_a}^2\le2\Bigr\} \] has probability tending to one. On $\Omega_N$ the Cauchy--Schwarz inequality gives, for every $\sigma\in S_N$, \begin{equation}\label{I:gap:opnorm} \frac1N\sum_{a\le M}\abs{b_a\cdot\sigma} \le\frac{\sqrt M}N\norm{A\sigma} \le\sqrt{\alpha_N}\,C_{\mathrm{op}}. \end{equation} \section{Contacts at optimal configurations}\label{I:gap:contacts-sec} We first count the contacts at maximizers of $m_N$. The argument uses only the Gaussian matrix, and not the variational problem. Its probabilistic input is Wendel's theorem on random half-spaces, which we restate for Gaussian vectors. \begin{lemma}\label{I:gap:wendel} Let $M,N\ge1$ and let $g_1,\dots,g_M$ be independent standard Gaussian vectors in $\R^N$. Then \begin{equation}\label{I:gap:wendel-formula} \Prob\bigl\{\text{there is }y\in\R^N\text{ with }g_a\cdot y>0\text{ for all }a\le M\bigr\} =2^{1-M}\sum_{j=0}^{N-1}\binom{M-1}j . \end{equation} \end{lemma} \begin{proof} This is \cite[(1)]{Wendel1962}. Wendel proved it for random vectors $x_1,\dots,x_M$ in $\R^N$ whose joint law is invariant under the reflection of each $x_a$ through the origin and such that, with probability one, every $N$ of them are linearly independent. The probability computed there is that of the event that the inner products of all $x_a$ with some vector $y$ are positive. Both hypotheses hold here. The $g_a$ are independent and have symmetric laws. For every set of $N$ rows, the determinant of the corresponding $N\times N$ matrix is a nonzero polynomial in its $N^2$ independent Gaussian entries, and therefore vanishes with probability zero. See also \cite[Theorem~1 and~(20)]{Cover1965}. \end{proof} The right side of \eqref{I:gap:wendel-formula} is the probability that a binomial variable with parameters $M-1$ and $1/2$ is at most $N-1$. For $M/N\to\alpha>2$ it tends to zero exponentially fast, and this is the only place where $\alpha>2$ enters the count of contacts. To describe the maximizers we use the following objects. For a set $S$ of $N$ rows such that $\{g_a:a\in S\}$ is linearly independent, let $y_S\in\R^N$ be the unique solution of $g_a\cdot y_S=-1$ for $a\in S$, and put \begin{equation}\label{I:gap:basis} u_S=\frac{y_S}{\norm{y_S}},\qquad k_S=-\frac1{\norm{y_S}} . \end{equation} Recall that a contact of $\sigma$ is a row $a$ with $b_a\cdot\sigma=m_N(\sigma)$. \begin{proposition}\label{I:gap:contacts} Let $\alpha>2$. \begin{enumerate}[label=\textup{(\roman*)}] \item For every $c<(\alpha-2)^2/(2\alpha)$ and all sufficiently large $N$, \begin{equation}\label{I:gap:kappaneg} \Prob\{\kappa_N\ge0\}\le\e^{-cN}. \end{equation} \item Almost surely the following holds. If $\kappa_N<0$, then every maximizer $\sigma_*$ of $m_N$ has exactly $N$ contacts, the set $S$ of its contacts has linearly independent rows $\{g_a:a\in S\}$, and, with the notation \eqref{I:gap:basis}, \begin{equation}\label{I:gap:vertex} \sigma_*=\sqrt N\,u_S,\qquad\kappa_N=k_S . \end{equation} \item Let $0\alpha\log3/\log(\sqrt{2\pi}/t)$. With probability tending to one, every maximizer $\sigma_*$ of $m_N$ satisfies \begin{equation}\label{I:gap:nearcontact} \#\bigl\{a\le M:02$, we have $M\ge N$ for large $N$, and we assume $M\ge N$ from now on. Let $\cG$ be the event on which every $N$ of the rows $g_a$ are linearly independent and, for every set $S$ of $N$ rows and every $a\notin S$, $g_a\cdot y_S\ne-1$. We claim that $\Prob(\cG)=1$. The first requirement holds almost surely by the determinant argument in the proof of \Cref{I:gap:wendel}. For the second, fix $S$ and $a\notin S$. Conditionally on the rows in $S$, the vector $g_a$ is standard Gaussian, so $g_a\cdot y_S$ is Gaussian with variance $\norm{y_S}^2>0$ and equals $-1$ with probability zero. A union over the finitely many pairs $(S,a)$ proves the claim. We divide the rest of the proof into four steps. \proofstep{1}{Proof of (i).} Suppose that $\cG$ holds and $\kappa_N\ge0$, and let $\sigma$ be a maximizer of $m_N$, so that $g_a\cdot\sigma\ge0$ for all $a$. Let $J=\{a:g_a\cdot\sigma=0\}$. The rows $g_a$, $a\in J$, lie in the $(N-1)$-dimensional space $\sigma^\perp$. On $\cG$ every $N$ rows are linearly independent, so $\abs J\le N-1$ and the rows in $J$ are linearly independent. Hence there is $v\in\R^N$ with $g_a\cdot v=1$ for all $a\in J$, and for small $\tau>0$ the vector $y=\sigma+\tau v$ satisfies $g_a\cdot y>0$ for all $a$. By \Cref{I:gap:wendel}, \[ \Prob\{\kappa_N\ge0\}\le\Prob\{Y_{M-1}\le N-1\}, \] where $Y_n$ denotes a binomial variable with parameters $n$ and $1/2$. For $00$, Markov's inequality and $\cosh x\le\e^{x^2/2}$ give \[ \Prob\{Y_n\le n/2-nh\}\le\e^{s(n/2-nh)}\E\e^{-sY_n} =\e^{-snh}\cosh(s/2)^n\le\e^{-snh+ns^2/8}, \] and the choice $s=4h$ gives $\Prob\{Y_n\le n/2-nh\}\le\e^{-2nh^2}$. Apply this with $n=M-1$ and $h_N=\frac12-\frac{N-1}{M-1}$, which is positive for large $N$ because $(N-1)/(M-1)\to1/\alpha<1/2$. Since \[ \frac{2(M-1)h_N^2}N\longrightarrow2\alpha\Bigl(\frac12-\frac1\alpha\Bigr)^2 =\frac{(\alpha-2)^2}{2\alpha}, \] this proves \eqref{I:gap:kappaneg}. \proofstep{2}{Maximizers as farthest points of a polytope.} Suppose that $\kappa_N<0$. Consider the polyhedron \[ K_N=\{y\in\R^N:g_a\cdot y\ge-1\text{ for all }a\le M\}, \] which is closed, convex, and contains $0$. It is bounded. Indeed, if $y_n\in K_N$ and $\norm{y_n}\to\infty$, then a limit point $d$ of $y_n/\norm{y_n}$ satisfies $\norm d=1$ and $g_a\cdot d\ge0$ for all $a$, and $\sigma=\sqrt Nd$ would have $m_N(\sigma)\ge0>\kappa_N$. For $\sigma\in S_N$ we have $m_N(\sigma)\le\kappa_N<0$, and $y=\sigma/(\sqrt N\abs{m_N(\sigma)})$ lies in $K_N$ with $\norm y=1/\abs{m_N(\sigma)}$. Conversely, if $y\in K_N\setminus\{0\}$, then $\sigma=\sqrt Ny/\norm y$ satisfies $m_N(\sigma)\ge-1/\norm y$, so that $\norm y\le1/\abs{m_N(\sigma)}\le1/\abs{\kappa_N}$. Hence $\max_{K_N}\norm y=1/\abs{\kappa_N}$, the farthest points of $K_N$ from the origin are exactly the points $y_*=\sigma_*/(\sqrt N\abs{\kappa_N})$ for maximizers $\sigma_*$, and $b_a\cdot\sigma_*=\kappa_N$ if and only if $g_a\cdot y_*=-1$. This description of the largest negative margin through the largest norm of a point of a polytope appears in \cite[Section~2.2]{MontanariZhongZhou2024}. A farthest point $y_*$ is an extreme point of $K_N$: if $y_*=\frac12(y_1+y_2)$ with $y_1\ne y_2$ in $K_N$, strict convexity gives $\norm{y_*}^2<\frac12(\norm{y_1}^2+\norm{y_2}^2)\le\max_{K_N}\norm y^2$. At an extreme point the active rows $\{g_a:g_a\cdot y_*=-1\}$ span $\R^N$. Otherwise some $v\ne0$ is orthogonal to all of them, the inactive rows satisfy $g_a\cdot y_*>-1$, and $y_*\pm\tau v\in K_N$ for small $\tau>0$. Hence every maximizer has at least $N$ contacts, and they contain $N$ linearly independent rows. The relation between extreme points and active constraints is standard in linear programming; see \cite[Theorem~2.3]{BertsimasTsitsiklis1997}. \proofstep{3}{Proof of (ii).} Suppose that $\cG$ holds and $\kappa_N<0$, let $\sigma_*$ be a maximizer, and let $y_*$ be as in Step~2. Choose a set $S$ of $N$ linearly independent contacts. Then $y_*=y_S$. If $\sigma_*$ had a further contact $a\notin S$, then $g_a\cdot y_S=-1$, which is excluded on $\cG$. Hence $S$ is the set of contacts of $\sigma_*$, and $\norm{y_S}=\norm{y_*}=1/\abs{\kappa_N}$ gives \eqref{I:gap:vertex}. \proofstep{4}{Proof of (iii).} Put $j_N=\lceil\eps N\rceil$. If $j_N>M-N$, then on $\cG\cap\{\kappa_N<0\}$ the asserted bound is automatic, since there are only $M-N$ noncontacts. We may therefore assume $j_N\le M-N$ in the counting argument below. Suppose that $\cG$ holds, $\kappa_N<0$, and some maximizer $\sigma_*$ violates \eqref{I:gap:nearcontact}. Let $S$ be its set of contacts. By \eqref{I:gap:vertex}, $b_a\cdot\sigma_*-\kappa_N=g_a\cdot u_S-k_S$ for every $a$. Hence there are a set $S$ of $N$ linearly independent rows and a set $J$ of $j_N$ rows outside $S$ such that $g_a\cdot u_S\in(k_S,k_S+t]$ for all $a\in J$. Fix $S$ and $J$. Conditionally on the rows in $S$, the unit vector $u_S$ and the number $k_S$ are fixed, and the variables $g_a\cdot u_S$, $a\notin S$, are independent standard Gaussian variables. Since the standard Gaussian density is at most $1/\sqrt{2\pi}$, the conditional probability of the event for $(S,J)$ is at most $(t/\sqrt{2\pi})^{j_N}$. The number of pairs $(S,J)$ is the multinomial coefficient $M!/(N!\,j_N!\,(M-N-j_N)!)$, which is at most $3^M$. By a union bound, by $t/\sqrt{2\pi}<1$, and by (i), \begin{align*} \Prob\{\text{some maximizer violates }\eqref{I:gap:nearcontact}\} &\le\Prob\{\kappa_N\ge0\}+3^M\Bigl(\frac t{\sqrt{2\pi}}\Bigr)^{\eps N}\\ &=\Prob\{\kappa_N\ge0\}+\exp\Bigl\{-N\Bigl(\eps\log\frac{\sqrt{2\pi}}t-\alpha_N\log3\Bigr)\Bigr\}. \end{align*} Both terms tend to zero, the second because $\alpha_N\to\alpha$ and $\eps\log(\sqrt{2\pi}/t)>\alpha\log3$. \end{proof} Parts (i) and (ii) show that, with probability at least $1-\e^{-cN}$, every optimal configuration is isostatic in the sense recalled in \Cref{I:sec:intro}: its $N$ contacts match the $N-1$ degrees of freedom on the sphere together with the optimized margin. In the replica computation of Franz, Parisi, Sevelev, Urbani, and Zamponi, isostaticity at jamming for $\kappa<0$ is derived from the scaling of the full replica symmetry breaking solution \cite[(100)]{FPSUZ2017}. Part (iii) is elementary but not sharp. In the proof of \Cref{I:thm:gaps} it gives the bound \eqref{I:gap:small-gaps} on the limiting gap law. \section{The finite recursion and the Parisi chain}\label{I:gap:recursion} Gap averages enter the Parisi functional through its terminal datum. For a step order parameter the Parisi equation is an explicit recursion, which accepts a hard wall with a source as terminal datum. We set up this recursion, define the Markov chain $\cQ$ that describes its linearization, and identify $\cQ$ with the Parisi diffusion. Let $\gamma\in\cU$ be a step order parameter. We write \begin{gather*} 0=t_0Q_\gamma$ and consecutive steps with equal values, so that further breakpoints may always be inserted. For a measurable terminal datum $F:\R\to[-\infty,\infty)$ that is bounded above we define $u_\gamma(\cdot,\cdot\,;F)$ at the breakpoints by $u_\gamma(1,x;F)=F(x)$ and, for $j=n,n-1,\dots,0$, \begin{equation}\label{I:gap:recursion-eq} u_\gamma(t_j,x;F)= \begin{cases} \dfrac1{m_j}\log\E\exp\bigl\{m_j\,u_\gamma\bigl(t_{j+1},x+\sqrt{\Delta_j}\,G;F\bigr)\bigr\}, & m_j>0,\\[3mm] \E\,u_\gamma\bigl(t_{j+1},x+\sqrt{\Delta_j}\,G;F\bigr), & m_j=0, \end{cases} \end{equation} where $G$ is a standard Gaussian variable. This is the recursion \eqref{I:pf:recursion} with terminal time one, evaluated at the breakpoints. Every function in the recursion is bounded above by $\sup F$. By \ref{I:pf:T2}, inserting breakpoints does not change $u_\gamma(t_j,\cdot\,;F)$ at the original breakpoints, and $u_\gamma(0,0;F)$ does not depend on the representation (\Cref{I:pf:step}). \paragraph{The hard wall.} For $F=H_k=\log\1_{[k,\infty)}$, \eqref{I:pf:Hkappa} with $\bar q=t_n$ gives \[ u^0_j:=u_\gamma(t_j,\cdot\,;H_k)=u_\gamma(t_j,\cdot\,;k),\qquad j\le n, \] the hard-wall solution computed with the cutoff $t_n$; in particular $u^0_n(x)=\log\Phi((x-k)/\sqrt{1-t_n})$. We put $u^0_{n+1}=H_k$. By \Cref{I:pf:apriori}(i), (iii), and (v), applied with $r=t_n$ and $\kappa=k$, each $u^0_j$ with $j\le n$ is smooth and concave on $\R$ (since $-\partial_{xx}u^0_j\ge0$ by (iii)), and $-C(1+x^2)\le u^0_j(x)\le0$. \paragraph{The source.} For a test function $\psi$ and $\beta\in\R$ we use the terminal datum $H_k+\beta\psi(\cdot-k)$ and write $u^\beta_j=u_\gamma(t_j,\cdot\,;H_k+\beta\psi(\cdot-k))$. The source $\beta\psi(\cdot-k)$ is bounded by $\abs\beta\norm\psi_\infty$, so \eqref{I:pf:monotone}, applied in both directions with $a=\abs\beta\norm\psi_\infty$, gives $\abs{u^\beta_j-u^0_j}\le\abs\beta\norm\psi_\infty$ for $j\le n$. In particular every $u^\beta_j$ with $j\le n$ is finite. For a terminal datum $F$ as above we put \[ \cP_F(\gamma)=\alpha u_\gamma(0,0;F)+\CS(\gamma),\qquad \cP_{k,\beta\psi}(\gamma)=\cP_{H_k+\beta\psi(\cdot-k)}(\gamma). \] Thus $\cP_U(\gamma)$ and $\cP_{k,0}(\gamma)=\cP_k(\gamma)$ are the functionals of \Cref{I:sec:model}. \paragraph{The chain.} Fix $k\in\R$ and let $p_\Delta(x,z)=(2\pi\Delta)^{-1/2}\e^{-(z-x)^2/(2\Delta)}$ be the Gaussian kernel. Let $\cQ$ be the law of the Markov chain $X_{t_0},X_{t_1},\dots,X_{t_n},X_1$ with $X_{t_0}=0$ and transitions \begin{equation}\label{I:gap:chain} \cQ_j(\dd z\mid x)=p_{\Delta_j}(x,z)\, \exp\bigl\{m_ju^0_{j+1}(z)-m_ju^0_j(x)\bigr\}\dd z, \qquad j=0,\dots,n, \end{equation} from $X_{t_j}=x$ to $X_{t_{j+1}}$. We call $\cQ$ the \emph{Parisi chain}. By \eqref{I:gap:recursion-eq}, $\e^{m_ju^0_j(x)}=\int p_{\Delta_j}(x,z)\e^{m_ju^0_{j+1}(z)}\dd z$, so each $\cQ_j(\cdot\mid x)$ is a probability measure. If $m_j=0$, it is the Gaussian law $N(x,\Delta_j)$. The measure $\cQ_j(\cdot\mid x)$ is carried by $I_j$, where $I_j=\R$ for $j0$, the Parisi diffusion is the Doob transform \cite{Doob1957,RevuzYor1999} of Brownian motion by the space-time harmonic function $\e^{mu_\gamma}$. Part (a) also shows that the law of $X^{\gamma,k}_1$ may be computed with any cutoff $r\ge Q_\gamma$, which is used for general order parameters. \begin{lemma}\label{I:gap:diffusion} Let $\gamma\in\cU$, $k\in\R$, and $r\in[Q_\gamma,1)$, and let $u=u_\gamma(\cdot,\cdot\,;k)$ be computed with the cutoff $r$. Let $X$ solve $\dd X_q=\gamma(q)\,\partial_xu(q,X_q)\dd q+\dd B_q$ on $[0,r]$ with $X_0=0$, for a standard Brownian motion $B$, and let $X_1$ be distributed, given $(X_q)_{q\le r}$, as $X_r+\sqrt{1-r}\,G$ conditioned on $\{X_r+\sqrt{1-r}\,G\ge k\}$. \begin{enumerate}[label=\textup{(\alph*)}] \item $((X_q)_{q\le Q_\gamma},X_1)$ has the law of the Parisi diffusion $X^{\gamma,k}$. \item If $\gamma$ is a step order parameter represented as above with $t_n=r$, then $(X_{t_0},\dots,X_{t_n},X_1)$ has the law $\cQ$. In particular, for every representation of $\gamma$, $X_1$ has the same law under $\cQ$ as $X^{\gamma,k}_1$. \end{enumerate} \end{lemma} \begin{proof} The drift $\gamma(q)\,\partial_xu(q,x)$ is Lipschitz in $x$, uniformly in $q$, and has linear growth. This is \Cref{I:pf:apriori}(iii), which extends to all of $\cU_r$ by \Cref{I:pf:coef-cont}. Hence the equation has a unique strong solution from every starting point. In particular solutions are unique in law, and $X$ is a Markov process. We use the following fact. \proofpart{Doob transform.} Let $[a,b]\subseteq[0,r]$ and $m>0$ be such that $\gamma=m$ on $[a,b)$, and suppose that $w=\e^{mu}$ satisfies $w(q,x)=\int p_{b-q}(x,z)w(b,z)\dd z$ on $[a,b]\times\R$. Then, given $X_a=x$, the variable $X_b$ has density $p_{b-a}(x,z)w(b,z)/w(a,x)$. Indeed, let $B$ be a Brownian motion started from $x$ at time $a$. Since $u\le0$ (\Cref{I:pf:apriori}(i)), we have $00$, then $w=\e^{m_ju}$ satisfies the hypothesis of the Doob transform on $[t_j,t_{j+1}]$ by the recursion \eqref{I:pf:recursion}, so the transition from $X_{t_j}$ to $X_{t_{j+1}}$ is $\cQ_j$. If $m_j=0$, the drift vanishes on $[t_j,t_{j+1})$ and the transition is $N(x,\Delta_j)=\cQ_j(\cdot\mid x)$. The last transition is $\cQ_n$ by construction. By the Markov property, $(X_{t_0},\dots,X_{t_n},X_1)$ has the law $\cQ$. The last assertion follows from (a). \end{proof} Accordingly, for a step order parameter $\gamma$ we write $\E_\gamma$ for expectation under the Parisi chain $\cQ$. By \Cref{I:gap:diffusion}(b), expectations of functions of $X_1$ under $\cQ$ agree with those under the Parisi diffusion $X^{\gamma,k}$. A general order parameter is handled by approximation with step order parameters that share its cutoff: every $\gamma\in\cU_r$ is the $L^1([0,1])$-limit of step order parameters in $\cU_r$ (\Cref{I:pf:step}). \begin{lemma}\label{I:gap:cutoff} Fix $r\in[0,1)$ and $k\in\R$. If $\gamma_i,\gamma\in\cU_r$ and $\gamma_i\to\gamma$ in $L^1([0,1])$, then \[ \cP_k(\gamma_i)\to\cP_k(\gamma),\qquad \lambda_{\gamma_i}(0)\to\lambda_\gamma(0),\qquad \E\varphi(X^{\gamma_i,k}_1)\to\E\varphi(X^{\gamma,k}_1) \] for every bounded continuous $\varphi:\R\to\R$. \end{lemma} \begin{proof} The first limit is the continuity of $\cP_k$ on $\cU_r$ in \Cref{I:pf:coef-cont}, and $\abs{\lambda_{\gamma_i}(0)-\lambda_\gamma(0)}\le\norm{\gamma_i-\gamma}_{L^1}$. For the third, let $X^{\gamma_i}$ and $X^\gamma$ be the diffusions of \Cref{I:gap:diffusion} with cutoff $r$, driven by the same Brownian motion. By \Cref{I:gap:diffusion}(a), \[ \E\varphi(X^{\gamma,k}_1)=\E\,\pi_\varphi(X^\gamma_r),\qquad \E\varphi(X^{\gamma_i,k}_1)=\E\,\pi_\varphi(X^{\gamma_i}_r),\qquad \pi_\varphi(x)=\frac{\int_k^\infty\varphi(z)p_{1-r}(x,z)\dd z}{\Phi((x-k)/\sqrt{1-r})}. \] The numerator of $\pi_\varphi$ is continuous in $x$ by dominated convergence, and the denominator is continuous and positive, so $\pi_\varphi$ is bounded and continuous. By \Cref{I:pf:coef-cont}, $\E\abs{X^{\gamma_i}_r-X^\gamma_r}^2\to0$. Hence $X^{\gamma_i}_r\to X^\gamma_r$ in probability, and $\E\pi_\varphi(X^{\gamma_i}_r)\to\E\pi_\varphi(X^\gamma_r)$ by bounded convergence. \end{proof} \section{A quadratic source estimate}\label{I:gap:transport} For configurations of nearly maximal margin the source strength will be large compared with the errors we must control, so we need the dependence of $\cP_{k,\beta\psi}(\gamma)$ on $\beta$ beyond first order, with a constant proportional to $\lambda_\gamma(0)$. Write $w_j=u^\beta_j-u^0_j$. By \eqref{I:gap:recursion-eq} and \eqref{I:gap:chain}, $w_j(x)$ equals $m_j^{-1}\log\int\e^{m_jw_{j+1}}\dd\cQ_j(\cdot\mid x)$ when $m_j>0$: an iterated logarithmic moment along the Parisi chain. If each transition were the Gaussian law $N(x,\Delta_j)$, the logarithmic moment of $m_j\beta$ times an $L$-Lipschitz function would exceed $m_j\beta$ times its mean by at most $m_j^2\beta^2L^2\Delta_j/2$. After division by $m_j$, summing $m_j\beta^2L^2\Delta_j/2$ over $j$ would give the bound $\frac12\beta^2L^2\lambda_\gamma(0)$ by \eqref{I:gap:mass}. The transitions of the Parisi chain are Gaussian laws tilted by the log-concave weights $\e^{m_ju^0_{j+1}}$. The next lemma shows that they share two properties of Gaussian laws, and the proof of \Cref{I:obs-transport} shows that these properties suffice. The variance bound in (a), from which (b) follows, is a case of the Brascamp--Lieb inequality \cite{BrascampLieb1976}, and the sub-Gaussian bound in (a) also follows from Caffarelli's contraction theorem \cite{Caffarelli2000} and Gaussian concentration. In one dimension a direct proof is short. \begin{lemma}\label{I:gap:onestep} Let $\gamma\in\cU$ be a step order parameter, $k\in\R$, $0\le j\le n$, and $x\in\R$, and let $W$ have law $\cQ_j(\cdot\mid x)$. \begin{enumerate}[label=\textup{(\alph*)}] \item $\E\e^{t(W-\E W)}\le\e^{t^2\Delta_j/2}$ for every $t\in\R$. Consequently $\Var W\le\Delta_j$ and $\E(W-\E W)^4\le16\Delta_j^2$. \item If $f:I_j\to\R$ is bounded and $L$-Lipschitz, then $x\mapsto\int f\dd\cQ_j(\cdot\mid x)$ is $L$-Lipschitz on $\R$. \end{enumerate} \end{lemma} \begin{proof} Write $m=m_j$ and $\Delta=\Delta_j$. If $m=0$, then $j0$. \proofpart{Proof of (a).} Completing the square gives $\e^{tz}p_\Delta(x,z)=\e^{tx+t^2\Delta/2}p_\Delta(x+t\Delta,z)$. Since $\e^{mu^0_j(y)}=\int p_\Delta(y,z)\e^{mu^0_{j+1}(z)}\dd z$ for every $y\in\R$, by \eqref{I:gap:recursion-eq}, the definition \eqref{I:gap:chain} gives \begin{equation}\label{I:gap:laplace} \E\e^{tW}=\exp\bigl\{tx+\tfrac12t^2\Delta+mu^0_j(x+t\Delta)-mu^0_j(x)\bigr\}, \qquad t\in\R . \end{equation} In particular $W$ has exponential moments of all orders, and differentiating \eqref{I:gap:laplace} at $t=0$ gives $\E W=x+m\Delta\,\partial_xu^0_j(x)$. Since $u^0_j$ is concave, $u^0_j(x+t\Delta)-u^0_j(x)\le t\Delta\,\partial_xu^0_j(x)$, and therefore $\E\e^{t(W-\E W)}\le\e^{t^2\Delta/2}$. Put $Y=W-\E W$. Applying this bound at $t$ and at $-t$ gives $\E[\e^{tY}+\e^{-tY}-2]\le2(\e^{t^2\Delta/2}-1)$. Since $\e^y+\e^{-y}-2=2\sum_{i\ge1}y^{2i}/(2i)!$ is at least $y^2$ and at least $y^4/12$, we obtain $t^2\E Y^2\le2(\e^{t^2\Delta/2}-1)$, which gives $\Var W\le\Delta$ as $t\to0$, and, with $t^2=1/\Delta$, $\E Y^4\le24(\e^{1/2}-1)\Delta^2\le16\Delta^2$. \proofpart{Proof of (b).} By \eqref{I:gap:chain}, $\cQ_j(\dd z\mid x)$ is proportional to $\e^{xz/\Delta}\mathsf p(z)\dd z$ on $I_j$, with $\mathsf p(z)=\e^{-z^2/(2\Delta)+mu^0_{j+1}(z)}\le\e^{-z^2/(2\Delta)}$. The Gaussian bound on $\mathsf p$ justifies differentiation under the integral, which gives \[ \frac{\dd}{\dd x}\int f\dd\cQ_j(\cdot\mid x)=\frac1\Delta\Cov\bigl(f(W),W\bigr). \] With $W'$ an independent copy of $W$, $\Cov(f(W),W)=\frac12\E[(f(W)-f(W'))(W-W')]$, which is at most $\frac L2\E(W-W')^2=L\Var W\le L\Delta$ in absolute value, by (a). \end{proof} \begin{lemma}\label{I:obs-transport} Let $\gamma\in\cU$ be a step order parameter, $k\in\R$, and let $\psi$ be an $L$-Lipschitz test function. Put $g_\gamma=\alpha\E\psi(X^{\gamma,k}_1-k)$. For every $\beta\in\R$, \begin{equation}\label{I:gap:quadratic} \cP_{k,\beta\psi}(\gamma)\le\cP_k(\gamma)+\beta g_\gamma+\frac{\alpha L^2\beta^2}{2}\,\lambda_\gamma(0). \end{equation} \end{lemma} \begin{proof} Fix $\beta\in\R$. Since $\CS(\gamma)$ does not depend on $\beta$, it suffices to show that \[ D_\beta=u^\beta_0(0)-u^0_0(0)-\beta\E_\gamma\psi(X_1-k) \] satisfies $D_\beta\le\frac12L^2\beta^2\lambda_\gamma(0)$. Neither $D_\beta$ nor $\lambda_\gamma(0)$ depends on the representation of $\gamma$: the values $u^\beta_0(0)$ and $u^0_0(0)$ do not, as shown after \eqref{I:gap:recursion-eq}, and the law of $X_1$ under $\cQ$ does not, by \Cref{I:gap:diffusion}(b). Fix $\theta\in(0,1]$ and, by inserting breakpoints, choose a representation with $\Delta_j\le\theta$ for all $j$. All objects below refer to this representation. We divide the rest of the proof into three steps. \proofstep{1}{Conditional expectations.} Put $\psi_{n+1}(z)=\psi(z-k)$ on $I_n$ and $\psi_j(x)=\int\psi_{j+1}\dd\cQ_j(\cdot\mid x)$ for $j=n,\dots,0$. Then $\psi_j(x)$ is the conditional expectation of $\psi(X_1-k)$ given $X_{t_j}=x$ under $\cQ$, and $\psi_0(0)=\E_\gamma\psi(X_1-k)$. Each $\psi_j$ is bounded by $\norm\psi_\infty$, and by \Cref{I:gap:onestep}(b) and backward induction each $\psi_j$ is $L$-Lipschitz. \proofstep{2}{The recursion for the difference.} Put $w_j=u^\beta_j-u^0_j$ on $\R$ for $j\le n$, and $w_{n+1}=\beta\psi_{n+1}$ on $I_n$. Then $\abs{w_j}\le\abs\beta\norm\psi_\infty$ for all $j$. Let $j\le n$. On $I_j$ we have $u^\beta_{j+1}=u^0_{j+1}+w_{j+1}$, and for $j=n$ both $\e^{u^\beta_{n+1}}$ and $\e^{u^0_{n+1}}$ vanish outside $I_n$. Dividing \eqref{I:gap:recursion-eq} for $u^\beta_j$ by the same formula for $u^0_j$ and using \eqref{I:gap:chain} gives \begin{equation}\label{I:gap:w-recursion} \begin{split} w_j(x)&=\frac1{m_j}\log\int\e^{m_jw_{j+1}}\dd\cQ_j(\cdot\mid x)\quad(m_j>0),\\ w_j(x)&=\int w_{j+1}\dd\cQ_j(\cdot\mid x)\quad(m_j=0). \end{split} \end{equation} \proofstep{3}{Induction.} Put \[ \eps_\theta=\frac{\beta^2L^2}2+\frac83\abs\beta^3L^3\e^{2\abs\beta\norm\psi_\infty}\theta^{1/2}, \] and $\lambda_\gamma(t_{n+1})=\lambda_\gamma(1)=0$. We show by backward induction that \begin{equation}\label{I:gap:induction} w_j\le\beta\psi_j+\eps_\theta\,\lambda_\gamma(t_j)\qquad(j=n+1,n,\dots,0). \end{equation} For $j=n+1$ this holds with equality. Assume it for $j+1$, and recall from \eqref{I:gap:mass} that $\lambda_\gamma(t_j)=\lambda_\gamma(t_{j+1})+m_j\Delta_j$. If $m_j=0$, integrating against $\cQ_j(\cdot\mid x)$ gives $w_j\le\beta\psi_j+\eps_\theta\lambda_\gamma(t_{j+1})=\beta\psi_j+\eps_\theta\lambda_\gamma(t_j)$. Let $m_j>0$, let $W$ have law $\cQ_j(\cdot\mid x)$, and put $Y=\psi_{j+1}(W)-\psi_j(x)$, so that $\E Y=0$ and $\abs Y\le2\norm\psi_\infty$. By \eqref{I:gap:w-recursion} and the induction hypothesis, \[ w_j(x)\le\eps_\theta\lambda_\gamma(t_{j+1})+\beta\psi_j(x)+\frac1{m_j}\log\E\e^{m_j\beta Y}. \] We bound the last term by a second-order expansion. With $a=m_j\beta$, the inequalities $\log(1+y)\le y$ and $\e^y\le1+y+\frac12y^2+\frac16\abs y^3\e^{\abs y}$ give \[ \log\E\e^{aY}\le\frac{a^2}2\E Y^2+\frac{\abs a^3}6\e^{2\abs a\norm\psi_\infty}\E\abs Y^3 . \] Let $W'$ be an independent copy of $W$. The second moment is $\E Y^2=\frac12\E(\psi_{j+1}(W)-\psi_{j+1}(W'))^2\le L^2\Var W\le L^2\Delta_j$, by \Cref{I:gap:onestep}(a). For the third moment, $Y$ is the conditional expectation of $\psi_{j+1}(W)-\psi_{j+1}(W')$ given $W$. By Jensen's inequality, the inequality $(y-y')^4\le8(y^4+y'^4)$, and \Cref{I:gap:onestep}(a), \[ \E Y^4\le L^4\E(W-W')^4\le16L^4\E(W-\E W)^4\le256L^4\Delta_j^2, \] and hence $\E\abs Y^3\le(\E Y^2\,\E Y^4)^{1/2}\le16L^3\Delta_j^{3/2}$. Since $00$, \[ \Prob\bigl\{f_N(k,\beta)\le\cP_{k,\beta\psi}(\gamma)+\eps\bigr\}\longrightarrow1. \] \end{lemma} \begin{proof} The function $\beta\psi(\cdot-k)$ is bounded and Lipschitz on $\R$, and $\sum_a\beta\psi(b_a\cdot\sigma-k)=N\beta\Psi_N(\sigma,k)$. Hence $f_N(k,\beta)$ is the normalized logarithm of the integral in \Cref{I:up-hardvolume}(ii) with the margin $k$ and the source $\beta\psi(\cdot-k)$. The value $\cP_{H_k+\beta\psi(\cdot-k)}(\gamma)$ there is defined by the recursion \eqref{I:pf:recursion} with the terminal datum $H_k+\beta\psi(\cdot-k)$. Evaluated at the breakpoints this is \eqref{I:gap:recursion-eq}, so it equals $\cP_{k,\beta\psi}(\gamma)$, and it does not depend on the representation of $\gamma$. \Cref{I:up-hardvolume}(ii) is therefore the claim. \end{proof} We now state the deviation inequality that both gap theorems use. It holds for every order parameter; the step approximation of \Cref{I:gap:cutoff} is performed inside the proof. \begin{proposition}\label{I:gap:master} Let $k\in\R$, $\gamma\in\cU$, $\beta>0$, and $\eps>0$, let $\psi$ be an $L$-Lipschitz test function, and put $g=\alpha\E\psi(X^{\gamma,k}_1-k)$. With probability tending to one, every Borel set $T\subseteq S_N(k)$ with $\mu_N(T)>0$ satisfies \begin{equation}\label{I:gap:master-eq} \bigl|\langle\Psi_N(\sigma,k)\rangle_T-g\bigr| \le\frac{\cP_k(\gamma)-N^{-1}\log\mu_N(T)}\beta +\frac{\alpha L^2\beta}2\,\lambda_\gamma(0)+\eps . \end{equation} \end{proposition} \begin{proof} We divide the proof into two steps: we first treat step order parameters and then approximate. \proofstep{1}{Step order parameters.} Let $\gamma$ be a step order parameter. For every Borel $T\subseteq S_N(k)$ with $\mu_N(T)>0$, restricting the integral defining $f_N(k,\beta)$ to $T$ and applying Jensen's inequality to $\langle\e^{N\beta\Psi_N(\sigma,k)}\rangle_T$ gives \[ f_N(k,\beta)\ge\frac1N\log\mu_N(T)+\beta\langle\Psi_N(\sigma,k)\rangle_T . \] By \Cref{I:gap:diffusion}(b), $g=\alpha\E_\gamma\psi(X_1-k)$. By \Cref{I:gap:source-upper} with $\beta\eps$ in place of $\eps$, and by \Cref{I:obs-transport}, with probability tending to one, \[ f_N(k,\beta)\le\cP_{k,\beta\psi}(\gamma)+\beta\eps \le\cP_k(\gamma)+\beta g+\frac{\alpha L^2\beta^2}2\,\lambda_\gamma(0)+\beta\eps . \] On this event, which does not depend on $T$, the two inequalities give \[ \langle\Psi_N(\sigma,k)\rangle_T-g\le\frac{\cP_k(\gamma)-N^{-1}\log\mu_N(T)}\beta +\frac{\alpha L^2\beta}2\,\lambda_\gamma(0)+\eps \] for every such $T$. The test function $-\psi$ is also $L$-Lipschitz, and replacing $\psi$ by $-\psi$ replaces $\Psi_N$ by $-\Psi_N$ and $g$ by $-g$. The same argument therefore bounds $g-\langle\Psi_N(\sigma,k)\rangle_T$ on a second event whose probability tends to one. The intersection of the two events has probability tending to one. \proofstep{2}{General order parameters.} Choose $r\in[Q_\gamma,1)$ and step order parameters $\gamma_i\in\cU_r$ with $\gamma_i\to\gamma$ in $L^1([0,1])$. By \Cref{I:gap:cutoff}, applied with the bounded continuous function $\varphi=\psi(\cdot-k)$, we have $\cP_k(\gamma_i)\to\cP_k(\gamma)$, $\lambda_{\gamma_i}(0)\to\lambda_\gamma(0)$, and $g_i=\alpha\E\psi(X^{\gamma_i,k}_1-k)\to g$. Fix $i$ such that \[ \frac{\abs{\cP_k(\gamma_i)-\cP_k(\gamma)}}\beta +\frac{\alpha L^2\beta}2\abs{\lambda_{\gamma_i}(0)-\lambda_\gamma(0)} +\abs{g_i-g}\le\frac\eps2 . \] Step~1 applied to $\gamma_i$ with $\eps/2$ in place of $\eps$ proves the proposition for $\gamma$. \end{proof} El Alaoui and Sellke described the empirical law of the fields along the iterates of their algorithm through the Parisi diffusion \cite[Section~3]{ElAlaouiSellke2022}. Empirical laws of the projections $b_a\cdot\sigma$ along data-dependent directions $\sigma$ were also studied by Montanari and Zhou \cite{MontanariZhou2024}; see also \cite{HuangSellkeSun2026}. Montanari and Zhou characterized the laws attainable by a class of efficient algorithms through a stochastic control problem. They passed between these laws and the largest averages of test functions by duality, and obtained a variational principle that extends the Parisi formula. In \Cref{I:gap:master} the test function enters the terminal datum of the Parisi functional as a source, and the bound applies to every set of positive volume in the feasible set. \section{Gaps of typical feasible configurations}\label{I:gap:proof-gibbs} We apply \Cref{I:gap:master} at a fixed margin $\kappa<\kc$, with a minimizer $\gamma$ of $\cP_\kappa$, to the sets of configurations whose gap average deviates from $\alpha\E\psi(X^{\gamma,\kappa}_1-\kappa)$. The hard-volume lower bound of \Cref{I:thm:capacity}(i) controls the volume term. \begin{proof}[Proof of \Cref{I:thm:gibbs-gaps}] Let $\alpha>2$ and $\kappa<\kc(\alpha)$. We divide the proof into three steps. \proofstep{1}{Preliminaries.} By \Cref{I:fun:critical}, $\cP_*(\kappa)>-\infty$. Hence minimizers of $\cP_\kappa$ exist (\Cref{I:fun:existence}). By \Cref{I:thm:capacity}(i), $V_N(\kappa)>0$, so that $\langle\cdot\rangle_{N,\kappa}$ is well defined, with probability tending to one. Let $\gamma$ be a minimizer of $\cP_\kappa$, so that $\cP_\kappa(\gamma)=\cP_*(\kappa)$, and put $\lambda=\lambda_\gamma(0)$. Then $\lambda\ge1-Q_\gamma>0$. Let $\psi$ be an $L$-Lipschitz test function with $L>0$, put $g=\alpha\E\psi(X^{\gamma,\kappa}_1-\kappa)$, and fix $\eps>0$. \proofstep{2}{Deviation sets.} Put $\beta=\eps/(2\alpha L^2\lambda)$, so that $\frac12\alpha L^2\beta\lambda=\eps/4$. Let $\cE_N$ be the event on which the conclusion of \Cref{I:gap:master} holds for the margin $\kappa$, the minimizer $\gamma$, this $\beta$, and $\eps/8$ in place of $\eps$, and on which \[ \frac1N\log V_N(\kappa)\ge\cP_*(\kappa)-\frac{\beta\eps}8 . \] By \Cref{I:gap:master} and \Cref{I:thm:capacity}(i), $\Prob(\cE_N)\to1$. The function $\sigma\mapsto\Psi_N(\sigma,\kappa)$ is continuous, so the sets \[ T_\pm=\bigl\{\sigma\in S_N(\kappa):\pm\bigl(\Psi_N(\sigma,\kappa)-g\bigr)>\eps\bigr\} \] are Borel. Suppose that $\cE_N$ holds and $\mu_N(T_+)>0$. Every $\sigma\in T_+$ has $\Psi_N(\sigma,\kappa)-g>\eps$, and $\mu_N(T_+)>0$, so $\langle\Psi_N(\sigma,\kappa)\rangle_{T_+}-g>\eps$. By \eqref{I:gap:master-eq} with $T=T_+$, \[ \eps<\frac{\cP_*(\kappa)-N^{-1}\log\mu_N(T_+)}\beta+\frac\eps4+\frac\eps8, \qquad\text{that is,}\qquad \frac1N\log\mu_N(T_+)<\cP_*(\kappa)-\frac{5\beta\eps}8 . \] Together with the lower bound on $V_N(\kappa)$ in the definition of $\cE_N$, this gives $\mu_N(T_+)<\e^{-N\beta\eps/2}V_N(\kappa)$. The same bound holds for $T_-$, and it also holds if $\mu_N(T_\pm)=0$. Since $\beta\eps/2=\eps^2/(4\alpha L^2\lambda)$, on $\cE_N$ \[ \mu_N\bigl(\{\sigma\in S_N(\kappa):\abs{\Psi_N(\sigma,\kappa)-g}>\eps\}\bigr) \le2\exp\Bigl\{-\frac{\eps^2N}{4\alpha L^2\lambda}\Bigr\}V_N(\kappa) \le\exp\Bigl\{-\frac{\eps^2N}{8\alpha L^2\lambda}\Bigr\}V_N(\kappa) \] as soon as $\eps^2N\ge8\alpha L^2\lambda\log2$. This proves \eqref{I:gap:gibbs-concentration}. \proofstep{3}{Gibbs averages and the law of $X_1$.} On $\cE_N$, since $\abs{\Psi_N(\sigma,\kappa)}\le\bar\alpha\norm\psi_\infty$ and $\abs g\le\alpha\norm\psi_\infty$, Step~2 gives, for large $N$, \[ \bigl|\langle\Psi_N(\sigma,\kappa)\rangle_{N,\kappa}-g\bigr| \le\eps+(\bar\alpha+\alpha)\norm\psi_\infty \exp\Bigl\{-\frac{\eps^2N}{8\alpha L^2\lambda}\Bigr\}\le2\eps . \] Since $\eps>0$ is arbitrary, $\langle\Psi_N(\sigma,\kappa)\rangle_{N,\kappa}\to g$ in probability. The averages $\langle\Psi_N(\sigma,\kappa)\rangle_{N,\kappa}$ do not depend on $\gamma$, and a limit in probability is unique. Hence $\E\psi(X^{\gamma,\kappa}_1-\kappa)$ is the same for all minimizers $\gamma$, for every bounded Lipschitz $\psi$. The variable $X^{\gamma,\kappa}_1-\kappa$ takes values in $[0,\infty)$, and bounded Lipschitz functions determine a finite Borel measure on $[0,\infty)$: for a closed set $F$ the functions $(1-j\operatorname{dist}(\cdot,F))_+$, $j\ge1$, are bounded and Lipschitz and decrease to $\1_F$, and a finite Borel measure is determined by its values on closed sets. Hence all minimizers induce the same law of $X^{\gamma,\kappa}_1$. \end{proof} In the replica computation of Franz, Parisi, Sevelev, Urbani, and Zamponi, the gap distribution is obtained as the derivative of the free energy with respect to a potential acting on the gaps, and it is expressed through the solution of the Fokker--Planck equation of the Parisi diffusion \cite[(37)]{FPSUZ2017}. The proof of \Cref{I:gap:master} perturbs the hard wall by a source and compares free energies in the same way. \Cref{I:thm:gibbs-gaps} is a rigorous counterpart of this description for the uniform measure on $S_N(\kappa)$ at a fixed margin $\kappa<\kc$. \section{Configurations near the optimal margin}\label{I:gap:cap} We turn to configurations of nearly maximal margin. To apply \Cref{I:gap:master} to one feasible configuration $\sigma_0$, we need a set of positive volume near $\sigma_0$ on which the fields $b_a\cdot\sigma$, and hence the gap averages, change little. \Cref{I:up-cap} provides such a set: a subset of a small cap around $\sigma_0$ that is feasible at a slightly smaller margin and has volume at least $\e^{-N(\log(1/\eta)+C)}$. This volume cost, of order $N\log(1/\eta)$ on the logarithmic scale, is what must be compared with the collapse of $\lambda_\gamma(0)$ near $\kc$. We record the consequence for gap averages. \begin{lemma}\label{I:obs-cap} Let $I$ be a compact interval. There are constants $N_0$ and $C_1\ge0$, depending only on $\bar\alpha$ and $I$, with the following property. Let $N\ge N_0$, suppose that $\Omega_N$ holds, and let $k\in I$, $\sigma_0\in S_N(k)$, and $0<\eta\le1$. There is a Borel set $T\subseteq S_N(k-\eta)$ with $-N^{-1}\log\mu_N(T)\le\log(1/\eta)+C_1$ such that, for every $L$-Lipschitz test function $\psi$ and every $\sigma\in T$, \begin{equation}\label{I:obs-cap-source} \abs{\Psi_N(\sigma,k-\eta)-\Psi_N(\sigma_0,k)}\le C_\psi\eta, \qquad C_\psi=L\bigl(\sqrt{\bar\alpha}\,C_{\mathrm{op}}+2\bar\alpha\bigr). \end{equation} \end{lemma} \begin{proof} Let $K_{\rm cap}\ge1$, $C_{\rm cap}$, and $N_0$ be the constants of \Cref{I:up-cap} for $\alpha'=\bar\alpha$, let $C_I$ be the constant there for the interval $I$, and put $C_1=\max\{C_{\rm cap}+C_I,0\}$. On $\Omega_N$ the rows satisfy $\norm{b_a}^2\le2$, and $M\le\bar\alpha N$, so \Cref{I:up-cap} applies to the margin $k$, the point $\sigma_0$, and this $\eta$. Since $k\in I$, it gives a Borel set $T\subseteq S_N(k-\eta)$ with $\mu_N(T)\ge\exp\{-N(\log(2/s_0)+C_{\rm cap})\}\ge\exp\{-N(\log(1/\eta)+C_1)\}$ and \[ \abs{b_a\cdot\sigma-b_a\cdot\sigma_0}\le s_0^2\abs{b_a\cdot\sigma_0}+s_0K_{\rm cap}, \qquad\sigma\in T,\ a\le M, \] where $s_0\le\eta/(2K_{\rm cap})$ by the definition of $s_0$ there. As $K_{\rm cap}\ge1$ and $\eta\le1$, we have $s_0K_{\rm cap}\le\eta/2$ and $s_0^2\le\eta^2/4\le\eta/4$. Since $\psi$ is $L$-Lipschitz, for $\sigma\in T$ and $a\le M$, \[ \abs{\psi(b_a\cdot\sigma-k+\eta)-\psi(b_a\cdot\sigma_0-k)} \le L\bigl(\abs{b_a\cdot\sigma-b_a\cdot\sigma_0}+\eta\bigr) \le L\Bigl(\frac\eta4\abs{b_a\cdot\sigma_0}+\frac{3\eta}2\Bigr). \] Averaging over $a$, and using \eqref{I:gap:opnorm} for $\sigma_0$ and $\alpha_N\le\bar\alpha$, gives $\abs{\Psi_N(\sigma,k-\eta)-\Psi_N(\sigma_0,k)} \le L\eta\bigl(\frac14\sqrt{\bar\alpha}\,C_{\mathrm{op}}+\frac32\bar\alpha\bigr) \le C_\psi\eta$. \end{proof} We now combine the cap with \Cref{I:gap:master}. The following proposition separates the probabilistic inputs, which are the convergence of $\kappa_N$ and the bounds already proved, from the deterministic input, which concerns only the variational problem: along some sequence of margins $k_n\uparrow\kc$ there are order parameters whose mass $\lambda_\gamma(0)$ is small compared with the volume cost $\log(1/(\kc-k_n))$ of the cap. \begin{proposition}\label{I:obs-optimizer} Assume that $\kappa_N\to\kc$ in probability, and let $\psi$ be a test function. Suppose that there are margins $k_n<\kc$ with $k_n\to\kc$ and order parameters $\gamma_n\in\cU$ such that \begin{equation}\label{I:gap:deterministic-condition} \lambda_{\gamma_n}(0)\Bigl(1+\bigl[\cP_{k_n}(\gamma_n)\bigr]_++\log\frac1{\kc-k_n}\Bigr) \longrightarrow0 . \end{equation} Then $g_n=\alpha\E\psi(X^{\gamma_n,k_n}_1-k_n)$ converges to a limit $g(\psi)$, and for every $\eps>0$ there is $\rho>0$ such that \begin{equation}\label{I:gap:optimizer-conclusion} \Prob\Bigl\{\sup_{\sigma\in S_N:\ m_N(\sigma)\ge\kc-\rho} \bigl|\Psi_N(\sigma,m_N(\sigma))-g(\psi)\bigr|\le\eps\Bigr\}\longrightarrow1 . \end{equation} In particular, since $m_N(\sigma_*)=\kappa_N$ for every maximizer $\sigma_*$ of $m_N$ and $\Prob\{\kappa_N\ge\kc-\rho\}\to1$, the conclusion applies simultaneously to all maximizers. \end{proposition} \begin{proof} Let $\psi$ be $L$-Lipschitz with $L>0$ (a Lipschitz function is $L$-Lipschitz for every large $L$), and write $\lambda_n=\lambda_{\gamma_n}(0)>0$. Discarding finitely many $n$, we assume $\kc-k_n\le1$, and we put $\eta_n=(\kc-k_n)/2\le\frac12$, so that $k_n+\eta_n=\kc-\eta_n$. Let $N_0$ and $C_1$ be the constants of \Cref{I:obs-cap} for the interval $I=[\kc-1,\kc]$, let $C_\psi$ be as in \eqref{I:obs-cap-source}, and put \[ \ell_n=1+\Bigl[\cP_{k_n}(\gamma_n)+\log\frac1{\eta_n}+C_1\Bigr]_+, \qquad \omega_n=L(2\alpha\lambda_n\ell_n)^{1/2}+(C_\psi+2\bar\alpha L)\eta_n . \] We divide the proof into four steps. \proofstep{1}{Trapping at the margin $\kc-\eta_n$.} Fix $n$ and $\eps>0$, and put $\beta_n=(2\ell_n/(\alpha L^2\lambda_n))^{1/2}$, so that \[ \frac{\ell_n}{\beta_n}+\frac{\alpha L^2\beta_n}2\,\lambda_n=L(2\alpha\lambda_n\ell_n)^{1/2}. \] Consider the event on which $\Omega_N$ holds and the conclusion of \Cref{I:gap:master} holds for the margin $k_n$, the order parameter $\gamma_n$, the source strength $\beta_n$, and this $\eps$. Its probability tends to one. On this event, for $N\ge N_0$, let $\sigma_0\in S_N(\kc-\eta_n)$, and let $T$ be the set of \Cref{I:obs-cap} for the margin $k=\kc-\eta_n\in I$ and $\eta=\eta_n$. Then $T\subseteq S_N(k-\eta_n)=S_N(k_n)$ and $-N^{-1}\log\mu_N(T)\le\log(1/\eta_n)+C_1$, so that $\cP_{k_n}(\gamma_n)-N^{-1}\log\mu_N(T)\le\ell_n$. Averaging \eqref{I:obs-cap-source} over $T$ gives $\abs{\langle\Psi_N(\sigma,k_n)\rangle_T-\Psi_N(\sigma_0,\kc-\eta_n)}\le C_\psi\eta_n$. Therefore \eqref{I:gap:master-eq} gives \[ \sup_{\sigma_0\in S_N(\kc-\eta_n)}\abs{\Psi_N(\sigma_0,\kc-\eta_n)-g_n} \le L(2\alpha\lambda_n\ell_n)^{1/2}+C_\psi\eta_n+\eps . \] \proofstep{2}{Changing the reference margin.} Since $\psi$ is $L$-Lipschitz, $\abs{\Psi_N(\sigma,r)-\Psi_N(\sigma,r')}\le\alpha_NL\abs{r-r'}\le\bar\alpha L\abs{r-r'}$. With probability tending to one, $\kappa_N\le\kc+\eta_n$. Let $\sigma$ satisfy $m_N(\sigma)\ge\kc-\eta_n$. Then $\sigma\in S_N(\kc-\eta_n)$ and $0\le m_N(\sigma)-(\kc-\eta_n)\le\kappa_N-\kc+\eta_n\le2\eta_n$. Together with Step~1 this shows that, with probability tending to one, \begin{equation}\label{I:gap:trap} \sup_{\sigma\in S_N:\ m_N(\sigma)\ge\kc-\eta_n} \abs{\Psi_N(\sigma,m_N(\sigma))-g_n}\le\omega_n+\eps . \end{equation} \proofstep{3}{The errors vanish.} Since $\log(1/\eta_n)=\log2+\log(1/(\kc-k_n))$, $\log(1/(\kc-k_n))\ge0$, and $C_1\ge0$, \[ \ell_n\le(2+\log2+C_1)\Bigl(1+\bigl[\cP_{k_n}(\gamma_n)\bigr]_++\log\frac1{\kc-k_n}\Bigr). \] Hence $\lambda_n\ell_n\to0$ by \eqref{I:gap:deterministic-condition}. Since also $\eta_n\to0$, we have $\omega_n\to0$. \proofstep{4}{The centers converge.} Fix $n$ and $n'$. With probability tending to one, $\kappa_N\ge\kc-\min\{\eta_n,\eta_{n'}\}$, and then every maximizer $\sigma_*$ of $m_N$ lies in both sets in \eqref{I:gap:trap}, for $n$ and for $n'$. For large $N$ the three events intersect, which gives $\abs{g_n-g_{n'}}\le\omega_n+\omega_{n'}+2\eps$. Since $\eps$ is arbitrary, $\abs{g_n-g_{n'}}\le\omega_n+\omega_{n'}$, and $\omega_n\to0$. Thus $(g_n)$ is a Cauchy sequence. Let $g(\psi)$ be its limit. Then $\abs{g_n-g(\psi)}\le\omega_n$. Given $\eps>0$, choose $n$ with $2\omega_n\le\eps/2$ and put $\rho=\eta_n$. By \eqref{I:gap:trap} with $\eps/2$ in place of $\eps$, with probability tending to one, $\abs{\Psi_N(\sigma,m_N(\sigma))-g(\psi)}\le\omega_n+\eps/2+\abs{g_n-g(\psi)}\le\eps$ for all $\sigma$ with $m_N(\sigma)\ge\kc-\rho$. This proves \eqref{I:gap:optimizer-conclusion}. \end{proof} \section{Gaps of nearly optimal configurations}\label{I:gap:proof-gaps} We first identify the limits of gap averages of near-maximizers, one test function at a time, by applying \Cref{I:obs-optimizer} to minimizers. The collapse of minimizers, \Cref{I:fun:collapse-prop}, supplies its hypothesis. The measure $\nu_\alpha$ is then obtained by tightness, which we derive from the operator norm bound \eqref{I:gap:opnorm} at maximizers. Finally, \Cref{I:gap:contacts} gives the contacts and the small positive gaps at maximizers. \begin{proof}[Proof of \Cref{I:thm:gaps}] Let $\alpha>2$. By \Cref{I:thm:capacity}, $\kc<0$ and $\kappa_N\to\kc$ in probability. We divide the proof into six steps. \proofstep{1}{Identification along one sequence.} Fix a test function $\psi$ and margins $k_n<\kc$ with $k_n\to\kc$. By \Cref{I:fun:critical}, $\cP_*(k_n)>-\infty$, and by \Cref{I:fun:existence} there are minimizers $\gamma_n$ of $\cP_{k_n}$; fix any. By \eqref{I:hw:test}, $\cP_{k_n}(\gamma_n)=\cP_*(k_n)\le\alpha\log\Phi(-k_n)<0$, so that $[\cP_{k_n}(\gamma_n)]_+=0$. By \Cref{I:fun:collapse-prop}, applied with $\eta=\kc-k_n$, there are $C$ and $\eta_0$, depending only on $\alpha$, such that $\lambda_{\gamma_n}(0)\le C(\kc-k_n)^{1/(2+2\alpha)}$ as soon as $\kc-k_n\le\eta_0$ (the second bound in \eqref{I:hw:collapserates}). Since $x^{1/(2+2\alpha)}(1+\log(1/x))\to0$ as $x\downarrow0$, the condition \eqref{I:gap:deterministic-condition} holds. \Cref{I:obs-optimizer} shows that $\alpha\E\psi(X^{\gamma_n,k_n}_1-k_n)$ converges to a limit $g(\psi)$, and that for every $\eps>0$ there is $\rho>0$ with \begin{equation}\label{I:gap:uniform} \Prob\Bigl\{\sup_{\sigma\in S_N:\ m_N(\sigma)\ge\kc-\rho} \bigl|\Psi_N(\sigma,m_N(\sigma))-g(\psi)\bigr|\le\eps\Bigr\}\longrightarrow1 . \end{equation} Given two sequences of margins and minimizers, the sequence obtained by interleaving them is again of this form, so the two limits coincide. Thus $g(\psi)$ depends only on $\psi$ and $\alpha$. \proofstep{2}{Mean gap at maximizers.} Maximizers of $m_N$ exist by compactness, and $m_N(\sigma_*)=\kappa_N$. For $\sigma\in S_N$, by \eqref{I:gap:opnorm}, \[ \int x\,\nu_{N,\sigma}(\dd x)=\frac1N\sum_a\bigl(b_a\cdot\sigma-m_N(\sigma)\bigr) \le\sqrt{\alpha_N}\,C_{\mathrm{op}}+\alpha_N\abs{m_N(\sigma)} \] on $\Omega_N$. Put $C_2=\sqrt{2\alpha}\,C_{\mathrm{op}}+2\alpha(\abs\kc+1)$. With probability tending to one, $\Omega_N$ holds and $\abs{\kappa_N}\le\abs\kc+1$. Then, for $N$ so large that $\alpha_N\le2\alpha$, every maximizer satisfies $\int x\,\nu_{N,\sigma_*}(\dd x)\le C_2$, and hence \begin{equation}\label{I:gap:tail-max} \nu_{N,\sigma_*}((t,\infty))\le\frac{C_2}t\qquad(t>0). \end{equation} \proofstep{3}{The measure $\nu_\alpha$; parts (i) and (iii).} Let $k_n$ and $\gamma_n$ be as in Step~1, and let $\pi_n=\alpha\Law(X^{\gamma_n,k_n}_1-k_n)$, a measure on $[0,\infty)$ of mass $\alpha$. For $t>0$ let $\chi_t(x)=\min\{1,(x-t)_+\}$, a bounded Lipschitz function with $\1_{[t+1,\infty)}\le\chi_t\le\1_{(t,\infty)}$. We first show that $g(\chi_t)\le C_2/t$. Given $\eps>0$, let $\rho$ be as in \eqref{I:gap:uniform} for $\chi_t$. With probability tending to one, $\kappa_N\ge\kc-\rho$, the event in \eqref{I:gap:uniform} holds, and the conclusion of Step~2 holds. For large $N$ these events intersect, and for a maximizer $\sigma_*$ they give \[ g(\chi_t)\le\int\chi_t\dd\nu_{N,\sigma_*}+\eps\le\nu_{N,\sigma_*}((t,\infty))+\eps \le\frac{C_2}t+\eps . \] Hence $g(\chi_t)\le C_2/t$, and \[ \limsup_{n\to\infty}\pi_n([t+1,\infty))\le\lim_{n\to\infty}\int\chi_t\dd\pi_n =g(\chi_t)\le\frac{C_2}t . \] For every $\eps>0$, choose $t$ so that $C_2/t<\eps/2$. The tail masses above $t+1$ are then less than $\eps$ for all sufficiently large $n$. Enlarging the compact interval to control the finitely many remaining measures proves that $(\pi_n)$ is tight. Apply Prokhorov's theorem \cite{Prokhorov1956} to the probability measures $\pi_n/\alpha$ on the Polish space $[0,\infty)$ and rescale. Every subsequence has a further subsequence converging weakly to a measure $\pi$ of mass $\alpha$, and by Step~1, $\int\psi\dd\pi=g(\psi)$ for every test function $\psi$. Bounded Lipschitz functions determine a finite measure on $[0,\infty)$, as shown in the proof of \Cref{I:thm:gibbs-gaps}, so $\pi$ depends neither on the subsequence nor on the sequence $(k_n,\gamma_n)$. We call it $\nu_\alpha$. Then $\nu_\alpha([0,\infty))=\alpha$ and $\int\psi\dd\nu_\alpha=g(\psi)$ for every test function $\psi$. Along every such sequence, every subsequence of $(\pi_n)$ has a further subsequence converging weakly to $\nu_\alpha$, and hence $\pi_n\to\nu_\alpha$ weakly. This proves (iii). Since $\int\psi\dd\nu_{N,\sigma}=\Psi_N(\sigma,m_N(\sigma))$, \eqref{I:gap:uniform} is (i). \proofstep{4}{Weak convergence at maximizers.} Let $\psi$ be a bounded continuous function on $[0,\infty)$, and let $0<\eps\le1$. Choose $t\ge1$ with $C_2/t\le\eps$ and $\nu_\alpha((t,\infty))\le\eps$, and let $\chi(x)=\min\{1,(t+1-x)_+\}$. The function $\psi\chi$ is continuous and vanishes on $[t+1,\infty)$, hence uniformly continuous. Its piecewise linear interpolation on a sufficiently fine uniform grid of $[0,t+1]$, extended by zero, is a bounded Lipschitz function $\psi'$ with $\sup\abs{\psi\chi-\psi'}\le\eps$. Since $\abs{\psi(1-\chi)}\le\norm\psi_\infty\1_{(t,\infty)}$, every maximizer satisfies \begin{align*} \Bigl|\int\psi\dd\nu_{N,\sigma_*}-\int\psi\dd\nu_\alpha\Bigr| &\le\Bigl|\int\psi'\dd\nu_{N,\sigma_*}-\int\psi'\dd\nu_\alpha\Bigr| +\eps(\alpha_N+\alpha)\\ &\qquad+\norm\psi_\infty\bigl(\nu_{N,\sigma_*}((t,\infty))+\nu_\alpha((t,\infty))\bigr). \end{align*} By part (i), applied to $\psi'$, and since $\kappa_N\to\kc$, the first term on the right is at most $\eps$ for all maximizers with probability tending to one. By \eqref{I:gap:tail-max} and the choice of $t$, for $N$ so large that $\alpha_N\le\alpha+1$ the remaining terms are at most $(2\alpha+1)\eps+2\norm\psi_\infty\eps$ with probability tending to one. Hence, for every bounded continuous $\psi$, $\sup_{\sigma_*}\abs{\int\psi\dd\nu_{N,\sigma_*}-\int\psi\dd\nu_\alpha}\to0$ in probability, the supremum being over all maximizers. This is the weak convergence in (ii). \proofstep{5}{Contacts and the atom at zero.} By \Cref{I:gap:contacts}(i) and (ii), for every $c<(\alpha-2)^2/(2\alpha)$ and all sufficiently large $N$, with probability at least $1-\e^{-cN}$ every maximizer has exactly $N$ contacts. This is the first assertion of (ii). On this event $\nu_{N,\sigma_*}(\{0\})=1$ for every maximizer. Let $\eps>0$, and choose $t\in(0,\sqrt{2\pi})$ so small that $\alpha\log3/\log(\sqrt{2\pi}/t)<\eps$. By \Cref{I:gap:contacts}(iii), with probability tending to one every maximizer also satisfies $\nu_{N,\sigma_*}((0,t])<\eps$, and hence $\nu_{N,\sigma_*}([0,t])<1+\eps$. For $h>0$ let $\psi_h(x)=(1-x/h)_+$, a test function with $\1_{\{0\}}\le\psi_h\le\1_{[0,h)}$. With probability tending to one, $\int\psi_h\dd\nu_{N,\sigma_*}\ge\nu_{N,\sigma_*}(\{0\})=1$ and $\int\psi_t\dd\nu_{N,\sigma_*}\le\nu_{N,\sigma_*}([0,t])<1+\eps$. By part (i) at maximizers, $\int\psi_h\dd\nu_{N,\sigma_*}\to\int\psi_h\dd\nu_\alpha$ in probability for every $h>0$. Hence $\int\psi_h\dd\nu_\alpha\ge1$ for every $h>0$, and letting $h\downarrow0$ gives $\nu_\alpha(\{0\})\ge1$ by dominated convergence. Also $\nu_\alpha(\{0\})\le\int\psi_t\dd\nu_\alpha\le1+\eps$ for every $\eps>0$. Therefore $\nu_\alpha(\{0\})=1$. \proofstep{6}{Small positive gaps.} Let $t>0$ be a continuity point of $\nu_\alpha$. For $0\delta(t_1)$, and let $t'\le t_1$ be a continuity point of $\nu_\alpha$. By \Cref{I:gap:contacts}(iii) with $t_1$ in place of $t$, with probability tending to one every maximizer satisfies \[ \frac1N\#\{a\le M:0\delta(t_1)$, letting $\eps\downarrow\delta(t_1)$ gives $\nu_\alpha((0,t'])\le\delta(t_1)$. The continuity points of $\nu_\alpha$ are dense, because $\nu_\alpha$ has at most countably many atoms, so $\nu_\alpha((0,t_1))\le\delta(t_1)$ by continuity of measures along increasing unions. For $0