Conductive polymer composites (CPCs) play an important role in various industrial applications including anti-static materials, electromagnetic interference shielding, sensors, and so forth. However, the contradiction between high conductivity and mechanical properties, especially tensile strength, is hard to balance many CPCs. In this paper, a self-reinforced polypropylene (PP)/graphene composite with segregated structures was fabricated by a novel coating-reducing-pressing method. By controlling the hot-pressing parameters, a special structure consisting of both an interconnected conductive graphene network and an unmolten fiber phase in a PP matrix can be obtained. PP/graphene composites with such a structure showed not only excellent electric properties but also outstanding mechanical properties. A super-low electrical percolation threshold of 0.0043 vol % was reached for the composites because of an effective formation of the conductive graphene network. The conductivity of composites with only 1.85 wt % graphene reached to 4.09 S/m. The composites also showed good electromagnetic shielding effectiveness under an X band with the highest EMI of 29.3 dB at 10 GHz. The existence of the PP fiber phase contributed to the improved mechanical strength and toughness of the self-reinforcing conductive composites compared to the blending samples, which made it even stimuli-responsive for bending deformation. These results were believed helpful in the design and fabrication of a new kind of conductive self-reinforced polymer composites.