We pursue in this paper our study of approximations of values and E-saddle-point policies in dynamic zero-sum games. After extending the general theorem for approximation, we study zero-sum stochastic games with countable state space and unbounded immediate reward. We focus on the expected average payoff criterion. We use some tools developed in [M. M. Tidball and E. Altman, SIAM J. Control Optim., 34 (1996), pp. 311-328] to obtain the convergence of the values as well as the convergence of the epsilon saddle-point policies in various approximation problems. We consider several schemes of truncation of the state space (e.g., finite state approximation) and approximations of games with discount factor close to one for the game with expected average cost. We use the extension of the general theorem for approximation to study approximations in stochastic games with complete information. Finally, we consider the problem of approximating the sets of policies. We obtain some general results that we apply to a pursuit evasion differential game.