Understanding the structural response of polyelectrolyte brushes to variation in both intrinsic and external properties is highly relevant for their application as functionalized interfaces and components of nanodevices. Using coarse-grained simulations, we examine an aspect that is largely unexplored, namely the dielectric mismatch between the solvent and the substrate. We systematically study how this permittivity contrast alters the brush structure over a range of Bjerrum length, polymer charge density, counterion size and valency, salt concentration, polymer grafting density, and external electric field. In addition to the expected brush contraction near metallic substrates and expansion on low-permittivity substrates, we find various regimes where variation of the substrate properties qualitatively alters the brush response.