We develop stochastic optimal control results for nonlinear discrete-time systems driven by disturbances modeled by a Markov chain. A characterization and a computational procedure for a control law which maximizes a cost functional, related to expected time-to-violate specified constraints or to expected total yield before constraint violation occurs, are discussed. Such an optimal control law may be viewed as providing drift counteraction and is, therefore, referred to as drift counteraction stochastic optimal control. Two simulation examples highlight opportunities for applications of these results to hybrid electric vehicle (HEV) powertrain management and to oil extraction. (C) 2007 Elsevier Ltd. All rights reserved.