We study a class of noncooperative stochastic games with unbounded cost functions and an uncountable state space. It is assumed that the transition law is absolutely continuous with respect to some probability measure on the state space. Undiscounted stochastic games with expected average costs are considered first. It is shown under a uniform geometric ergodicity assumption that there exists a stationary epsilon-equilibrium for each epsilon > 0. The proof is based on recent results on uniform bounds for convergence rates of Markov chains [S. P. Meyn and R. L. Tweedie, Ann. Appl. Probab., 4 (1994), pp. 981 1011] and on an approximation method similar to that used in [ A. S. Nowak, J. Optim. Theory Appl., 45 (1985), pp. 591 602], where an epsilon-equilibrium in stationary policies was shown to exist for the bounded discounted costs. The stochastic game is approximated by one with a countable state space for which a stationary Nash equilibrium exists (see [E. Altman, A. Hordijk, and F. M. Spieksma, Math. Oper. Res., 22 (1997), pp. 588 618]); this equilibrium determines an epsilon-equilibrium for the original game. Finally, new results for the existence of stationary epsilon-equilibrium for discounted stochastic games are given.