math.ST — Statistics Theory

A Positive Dispersion Optimum for the Rounded Sample Mean: Sufficiency and Necessity of Finite Variance

Contributed by Mark R. Diamond, Angela O'Brien-Malone

A continuous quantity, observable only after nearest-lattice rounding, is estimated by the mean of $n$ rounded readings of an imperfect instrument. Averaged over the position of the true value within a rounding cell, the mean-squared error of this estimator is a function of one dimensionless parameter: the pre-rounding standard deviation $s$, measured in cell widths. A perfectly repeatable instrument ($s=0$) returns the same rounded value on every reading, so averaging provides no additional information; its cell-averaged error is the rounding floor $1/12$. We show that this floor is never optimal once more than one reading is taken. For \emph{any} centred pre-rounding law with finite, non-zero variance and any sample size $n>1$, some strictly positive dispersion yields a smaller cell-averaged error than zero, so the global minimizer lies away from the no-scatter endpoint; and each of the three hypotheses---$n>1$, finite variance, non-degeneracy---is individually necessary, so existence is \emph{exactly} characterized. The mechanism is elementary: the autocorrelation of the rounding error has a \emph{corner} at zero lag, so a little scatter relieves the rounding bias at first order in $s$ while its variance contribution grows only at second order. We give the short corner-based proof and then derive the spectral form of the criterion, using it to compute the optimum for four laws. The Gaussian optimizer is unique and solves a Jacobi-theta equation. For the Laplace law the bias term has a hyperbolic closed form, and the optimizer is unique and characterized by the corresponding derivative equation. For the uniform and two-point laws the optimizer is explicit. The uniform law cancels every bias harmonic exactly at its first positive cancellation scale, which saturates the natural localization bound when $n=2$; and the two-point law, whose criterion has several local wells once $n\ge4$, shows that finite variance ensures existence but not a single-well objective.