In this paper, distributed games for large-population multiagent systems with random time-varying parameters are investigated, where the agents are coupled via their individual costs and the structure parameters are a family of independent Markov chains with identical generators. The cost function of each agent is a long-run average tracking-type functional with an unknown mean field coupling nonlinear term as "reference signal." To reduce the computational complexity, the mean field approach is applied to construct distributed strategies. The population statistics effect (PSE) is used to approximate the average effect of all the agents, and the distributed strategies are given through solving a Markov jump tracking problem. Here the PSE is a deterministic quantity and can be obtained by solving the Stackelberg equilibrium of an auxiliary two-player game. It is shown that the closed-loop system is uniformly stable, and the distributed strategies are asymptotically optimal in the sense of Nash equilibrium, as the number of agents grows to infinity. A numerical example is provided to demonstrate the procedure of designing the strategies as well as the influence of the heterogeneity intensity and the parameter jump rate of the agents on the closed-loop system.