In this note, conditions are proven under which a real-time implementable moving horizon estimation (MHE) scheme is locally convergent. Specifically, the real-time iteration scheme of [17] is studied in which a single Gauss-Newton iteration is applied to approximate the solution to the respective MHE optimization problem at each time-step. Convergence is illustrated by a challenging small scale example, the Lorenz attractor with an unknown parameter. It is shown that the performance of the proposed real-time MHE algorithm is nearly identical to a fully converged MHE solution, while its fixed execution time per sample would allow one to solve 30 000 MHE problems per second on current hardware.