Many numerical schemes can be suitably studied from a system theoretic point of view. This paper studies the relationship between the two disciplines, that is, numerical analysis and system theory. We first see that various iterative solution schemes for linear and nonlinear equations can be suitably transformed into the form of a closed-loop feedback system, and show the crucial role of the internal model principle in such a context. This leads to new stability criteria for Newton's method. We then study Runge-Kutta type methods for solving differential equations, and also derive new stability criteria based on recent results on LMI. A numerical example is given to illustrate the advantage of the present theory. (c) 2007 Elsevier Ltd. All rights reserved.