In this paper we study the nonzero-sum constrained stochastic games for continuous-time jump processes with denumerable states and possibly unbounded transition rates. The optimality criterion under consideration is the expected average payoff criterion and the payoff functions of the players are allowed to be unbounded. Under the reasonable conditions, we introduce an approximating sequence of the auxiliary game models and show the existence of stationary constrained Nash equilibria for these approximating game models via employing the average occupation measures and constructing a suitable multifunction. Moreover, we obtain that any limit point of the stationary constrained Nash equilibria for the approximating sequence of the game models is a constrained Nash equilibrium for the original game model. Furthermore, we use a controlled birth and death system to illustrate our main results.