Monodisperse size crosslinked polymeric particles of specific chemical compositions, synthesized by emulsifier-free emulsion polymerization, were used as model fillers to study the effect of filler chemical composition on stress-strain behavior of rubbery composites. The modulus, E or G of filled composites increased while the stress and the strain at break decreased with increasing filler-matrix interactions. Physical crosslinking, either due to particle clustering or a network of filler particles with an adsorbed polymer layer supplemented chemical crosslinking. As a result, the overall crosslink density(chemical and physical) was effectively enhanced. The strength of the physical networks, and hence the stiffness of the composites increases with increasing particle-matrix interactions. However, excessively strong matrix-filler interaction would cause a loss of polymer flexibility at the particle-matrix interface, resulting in a decreased stress and elongation at break of the particle filled composites in the order PS > PMMA > PSVP.