Graphitic carbon nitride (g-C3N4), as an attractive photocatalyst, has been extensively modified for enhancing its photocatalytic activity. In the present work, the electronic structure, carrier separation efficiency, band edges position, and optical property of g-C3N4 catalyst modified by five-membered heterocyclic small molecules as a ligand (thiophene, furan, pyrrole, or pyrryl) have been investigated by means of the Density Functional Theory (DFT) calculations. The results show that the carrier migration and separation efficiency as well as the utilization of visible light were enhanced after all four types of modifications. However, considering the relationship of overpotential of water redox reaction over g-C3N4 and band edge positions with respect to the water redox potentials, only adsorbing pyrrole can meet the requirements of the over-potential of the photoelectrochemical (PEC) splitting of water to generate H-2 and O-2. Due to the more activating n-pi* electrons transitions in the distorted configurations, the optical absorption curves of the surface modified g-C3N4 systems are all red-shifted, while only the pyrrole modified system has two impurity energy levels, which indicates the rapid electron transit. Based on the above analysis, we propose that pyrrole should be an excellent ligand for the modification of g-C3N4 to improve the photocatalytic activity under visible light irradiation.