Due to its unique electronic properties, graphene has already been identified as a promising material for future carbon based electronics. To develop graphene technology, the fabrication of a high quality P-N junction is a great challenge. Here, we describe a general technique to grow single crystalline polyaniline (PANI) films on graphene sheets using in situ polymerization via the oxidation-reduction of aniline monomer and graphene oxide, respectively, to fabricate a high quality P-N junction, which shows diode-like behavior with a remarkably low turn-on voltage (60 mV) and high rectification ratio (1880: 1) up to a voltage of 0.2 V. The origin of these superior electronic properties is the preferential growth of a highly crystalline PANI film as well as lattice matching between the d-values [similar to 2.48 angstrom] of graphene and {120} planes of PANI.