A direct mapping of photovoltage in a complete Mo/CZTSSe/CdS/ZnO/Al:ZnO solar cell device is carried out using Kelvin probe force microscopic (KPFM) measurements in surface and junction modes. Four cells having different values of open circuit voltage (V-OC) have been studied, and nanoscale variation of photovoltage have been obtained from the difference of surface potential (SP) images in the two modes. The maps exhibit a higher photovoltage at grain boundaries in general. Observed SP image of pristine CZTSe layer reveals downward band bending resulting in reduction of carrier recombination, and thus, lower J(dark) at grain boundaries (GB). The observed downward band bending resulting in low Jdark at GB in CZTSe layer is used to explain the increase in photovoltage (V-OC) at the GB. KPFM measurements of CZTSe and CZTSe/CdS layers show that the surface potential of CZTSe layer and its variation can be affected by CdS deposition and further device processing, in addition to surface adsorption and contamination effects. The photovoltage mapping obtained from Kelvin probe force microscopic measurements on the final device without any interfering effects, is an important advantage of the present method.