Red or yellow phosphors excited by a blue light-emitting diode are an efficient source of white light for everyday applications. Many solid oxides and nitrides, particularly silicon nitride-based materials such as M2Si5N8 and MSi2O2N2 (M = Ca, Sr, Ba), CaAlSiN3, and SiAlON, are useful phosphor hosts with good thermal stabilities. Both oxide/nitride and various cation substitutions are commonly used to shift the emission spectrum and optimize luminescent properties, but the underlying mechanisms are not always clear. Here we show that size-mismatch between host and dopant cations tunes photoluminescence shifts systematically in M1.95Eu0.05Si5-xAlxN8-xOx lattices, leading to a red shift when the M = Ba and Sr host cations are larger than the Eu2+ dopant, but a blue shift when the M = Ca host is smaller. Size-mismatch tuning of thermal quenching is also observed. A local anion clustering mechanism in which Eu2+ gains excess nitride coordination in the M = Ba and Sr structures, but excess oxide in the Ca analogues, is proposed for these mismatch effects. This mechanism is predicted to be general to oxynitride materials and will be useful in tuning optical and other properties that are sensitive to local coordination environments.