In the epoxidation of ethylene, catalyzed by supported Ag particles, the support plays not only a beneficial role, but can also negatively impact the selectivity to ethylene oxide. Especially high surface area supports, and supports containing acid groups, seem detrimental for the selectivity, which is attributed to secondary reactions on the support surface. We prepared Ag nanoparticles on supports with a wide range of specific surface areas and different chemical nature, but with similar Ag metal loading and particle size. Indeed, a strong dependence of selectivity on both the specific surface area, and the chemical nature of the oxide support was found, with alpha-Al2O3 giving much more selective catalysts than SiO2. Furthermore, the conversion was an important factor in determining the selectivity, while on the other hand the support had no influence at all on the ethylene conversion. We described our experimental selectivities with a reaction model that used the rate constant for the secondary reaction of oxidation of ethylene oxide to carbon dioxide and water as the only variable parameter. The model was able to adequately describe the experimental results. It gave insight into the dependence of selectivity on conversion, how the secondary reaction depended on the chemical nature of the support, and how its rate scaled linearly with the support's specific surface area. Building on this insight we modified a SiO2 support to passivate its OH-groups, which indeed yielded a 94% decrease in the rate of the secondary reaction, performing even better than alpha-Al2O3 with similar specific surface area would. We have shown that the selectivity towards ethylene oxide is dependent on a metal specific, intrinsic selectivity, and the decrease in selectivity with conversion to be support dependent. This decrease is caused by the isomerization of ethylene oxide, which is found to be first order in ethylene oxide and the active sites on the catalyst support.