An effective ternary organic-inorganic composite flame retardant of reduced graphene oxide-poly-dopamine@graphitic carbon nitride@10-(2,5-dihydroxyphenyl)- 10-H-9-oxa-10-phosphaphenanthrene-10-oxide (RGO-PDA@g-C3N4@ODOPB) was successfully fabricated by co-precipitation method. Its property concerning the intrinsic flame retardancy and the mechanical performance was well studied when it was used as co-additives in combination with ammonium polyphosphate (APP) in epoxy resin (EP) samples. The surface morphology and the structure of RGO-PDA@g-C3N4 @ODOPB were characterized by SEM, and the molecular structure and compositions were investigated by FT-IR, powder XRD and 1H NMR. TGA, limit oxygen index (LOI), vertical burning test (UL-94), cone calorimeter test, and SEM were also used to investigate the thermal properties and flame retardancy of materials. As expected, the flame retardancy of EP was significantly heightened after adding of RGOPDA@ g-C3N4@ODOPB composites. It showed that with the 20% adding of RGO PDA@g C3N4@ODOPB/APP into EP led to the decreasing of the peak heat release rate and the total heat release at 78% and 62.5%, respectively. Meanwhile, the LOI value of the EP composites was as high as 29% and reached UL-94 V-0 rate. It was deemed that the excellent flame retardancy was attributed to the forming of compact and stable carbon layer, which was being catalytic carbonization by APP existed in the RGO PDA@g-C3N4@ODOPB/APP composites. At the same time, the non-combustible gas released from thermal cracking of g-C3N4 during the combustion also benefited the flame retardant performance of EP.