The usual understanding in polymer electrolyte design is that an increase in the polymer dielectric constant results in reduced ion aggregation and therefore increased ionic conductivity. We demonstrate here that in a class of polymers with extensive metal-ligand coordination and tunable dielectric properties, the extent of ionic aggregation is delinked from the ionic conductivity. The polymer systems considered here comprise ether, butadiene, and siloxane backbones with grafted imidazole sidechains, with dissolved Li+, Cu2+, or Zn2+ salts. The nature of ion aggregation is probed using a combination of X-ray scattering, electron paramagnetic resonance (in the case where the metal cation is Cu2+), and polymer field theory-based simulations. Polymers with less polar backbones (butadiene and siloxane) show stronger ion aggregation in X-ray scattering compared to those with the more polar ether backbone. The T-g-normalized ionic conductivities were however unaffected by the extent of aggregation. The results are explained on the basis of simulations which indicate that polymer backbone polarity does impact the microstructure and the extent of ion aggregation but does not impact percolation, leading to similar ionic conductivity regardless of the extent of ion aggregation. The results emphasize the ability to design for low polymer T-g through backbone modulation, separately from controlling ion-polymer interaction dynamics through ligand choice.