Reduced graphene oxide (RGO) loaded with magnetic Fe3O4 nanoparticles was synthesized through coprecipitation. Then, an Fe3O4/RGO-modified polyimide aqueous sizing agent was successfully synthesized with ultrasonic dispersion technology and coated on the surface of a high modulus carbon fiber (CF) under an applied magnetic field. The X-ray diffraction (XRD) and X-ray photoelectron spectroscopy (XPS) results show that the Fe3O4 with a cubic spinel crystal structure was successfully prepared. The morphology and particle size of the Fe3O4/RGO were characterized using transmission electron microscopy (TEM), indicating that Fe3O4 was uniformly dispersed onto the RGO sheet. Fourier transform infrared spectroscopy (FTIR) and a laser particle size analyzer proved that the aqueous dispersion of polyamic acid (PAA) was obtained with an average particle diameter of 158 nm. The mechanical property testing results showed that the interlaminar shear strength (ILSS) of CF/polyimide (PI) composites consisting of CFs coated with the Fe3O4/RGO-modified PI aqueous sizing agent under a magnetic field increased by 159.0%. This is attributed to the orientation of the Fe3O4 magnetic particles under a magnetic field that allowed them to be uniformly dispersed on the graphene, effectively weakening the potential for graphene agglomeration.