An infinite horizon stochastic optimal control problem with running maximum cost is considered. The value function is characterized as the viscosity solution of a second-order HamiltonJacobi-Bellman equation with mixed boundary condition. A general numerical scheme is proposed and convergence is established under the assumptions of consistency, monotonicity, and stability of the scheme. These properties are verified for a specific semi-Lagrangian scheme.