Pressure on ancillary reserves, i.e. frequency preserving, in power systems has significantly mounted due to the recent generalized increase of the fraction of (highly fluctuating) wind and solar energy sources in grid generation mixes. The energy storage associated with millions of individual customer electric thermal (heating cooling) loads is considered as a tool for smoothing power demand/generation imbalances. The piecewise constant level tracking problem of their collective energy content is formulated as a linear quadratic mean field game problem with integral control in the cost coefficients. The introduction of integral control brings with it a robustness potential to mismodeling, but also the potential of cost coefficient unboundedness. A suitable Banach space is introduced to establish the existence of Nash equilibria for the corresponding infinite population game, and algorithms are proposed for reliably computing a class of desirable near Nash equilibria. Numerical simulations illustrate the flexibility and robustness of the approach. (C) 2018 Elsevier Ltd. All rights reserved.