This article is about nonstationary nonlinear discrete-time deterministic and stochastic control systems with Borel state and control spaces, possibly noncompact control constraint sets, and unbounded costs. The control problem is to minimise an infinite-horizon total cost performance index. Using dynamic programming arguments we show that, under suitable assumptions, the optimal cost functions satisfy optimality equations, which in turn give a procedure to find optimal control policies.