Through the neutralization reaction of zinc oxide (ZnO) and methacrylic acid (MAA) or acrylic acid (AA), zinc methacrylate (ZMA) or zinc acrylate (ZA) was in situ prepared in nitrile rubber (NBR). The mechanical properties and crosslinking structure of the resulting peroxide-cured NBR vulcanizates were studied. The results showed that ZnO/MAA (AA) had a great reinforcing effect for NBR, and their amounts and ratio played important roles in influencing the mechanical properties. Such vulcanizate contains both covalent crosslinks and salt crosslinks, and the change in the tensile strength of the vulcanizate was related to the variation of the salt crosslink density.