A modified graphene oxide (MGO) was prepared by graphene oxide, boric acid, and 3-aminopropyltriethoxysilane in an ethanol aqueous solution using ultrasonic agitation. Its structure was characterized by the Fourier transform infrared Fourier transform infrared, X-ray diffraction, and elemental analysis. Subsequently, MGO as a synergist, accompanied with ammonium polyphosphate (APP), was incorporated into epoxy resins (EP) to improve its mechanical properties and flame retardancy. The flame retardancy of EP/APP/MGO nanocomposites was investigated by the limiting oxygen index (LOI) test, the UL 94 test and the cone calorimeter test (CCT). The LOI value of EP/APP/MGO nanocomposites reached 28.8% by addition of 15wt% of flame retardant, and its UL 94 test reached V-0 rating. The CCT results revealed that MGO could significantly enhance flame retardant properties of EP/APP/MGO nanocomposites because of the denser structure of char residue. Here, MGO also enhanced the mechanical properties of EP.