Nanoparticles significantly reinforce rubber and enhance nonlinear mechanical behavior while the underlining mechanisms are not clear. Herein the effects of crosslinking, filler content and filler surface chemistry on the linear and nonlinear rheological responses of silica filled nitrile butadiene rubber are investigated. A time-concentration superposition principle is employed for constructing rheological master curves. While the strain amplification and dynamics retardation contribute to the high-frequency linear rheology of the compounds, the former is crucial for the vulcanizates with matrix of similar crosslinking degree. In both the compounds and vulcanizates, the Payne effect is initiated at a similar microscopic strain of the rubber phase. A weak overshoot of loss modulus appears in compounds containing hydrophilic silica and vulcanizates containing hydrophobic silica. It is suggested that the viscoelastic matrix is highly important for the rheological responses of the filled compounds and vulcanizates.