Descent algorithms based on the gradient, conjugate gradient, and Newton methods are investigated to perform optimization in moving horizon state estimation for discrete-time linear and nonlinear systems. Conditions that ensure the stability of the estimation error are established for single and multi iteration schemes with a least-squares cost function that takes into account only a batch of most recent information. Simulation results show the effectiveness of the proposed approaches also in comparison with techniques based on the Kalman filter.