This work reported a novel magnetorheological plastomer doped with hard magnetic NdFeB particles (named as H-MRP). The dynamic properties of the H-MRP were systematically tested, and the mechanism of its unusual response to magnetic field was discussed. Unlike the MRP filled with soft magnetic particles (S-MRP), the storage modulus (G') of H-MRPs kept growing by 20% when the magnetic field decreased, while that of S-MRPs decreased with decreasing of the magnetic field immediately without an obvious change. Under the magnetic field ranging from 0 to 1 T, there was a peak value in the rising stage of G' at about 400 mT magnetic field, while that of S-MRPs changed monotonically. After an off-on progress of the magnetic field, the final G' of H-MRPs increased by over 10%. A possible mechanism was proposed to study the microstructure dependent mechanical properties. It was found that the complex viscoelastic behavior originated from the large hysteresis characteristics of the H-MRPs.