The trust-region self-consistent field (TRSCF) method is presented for optimizing the total energy E-SCF of Hartree-Fock theory and Kohn-Sham density-functional theory. In the TRSCF method, both the Fock/Kohn-Sham matrix diagonalization step to obtain a new density matrix and the step to determine the optimal density matrix in the subspace of the density matrices of the preceding diagonalization steps have been improved. The improvements follow from the recognition that local models to E-SCF may be introduced by carrying out a Taylor expansion of the energy about the current density matrix. At the point of expansion, the local models have the same gradient as E-SCF but only an approximate Hessian. The local models are therefore valid only in a restricted region-the trust region-and steps can only be taken with confidence within this region. By restricting the steps of the TRSCF model to be inside the trust region, a monotonic and significant reduction of the total energy is ensured in each iteration of the TRSCF method. Examples are given where the TRSCF method converges monotonically and smoothly, but where the standard DIIS method diverges. (C) 2004 American Institute of Physics.