The microscopic mechanism of polarization fatigue (i.e., a loss of switchable polarization under electrical cycling) remains one of the most important long-standing problems in ferroelectric communities. Although there are numerous proposed fatigue models, a consensus between the models and experimental results is not reached yet. By using modified-piezoresponse force microscopy, nanoscale domain switching dynamics are visualized for different fatigue stages in epitaxial PbZr0.4Ti0.6O3 capacitors. Systematic time-dependent studies of the domain nucleation and evolution reveal that domain wall pinning, rather than nucleation inhibition, is the primary origin of fatigue. In particular, the evolution of domain wall pinning process during electrical cycling, from the suppression of sideways domain growth in early fatigued stages to the blockage of forward domain growth in later stages, is directly observed. The pinning of forward growth results in a nucleation-limited polarization switching and a significant slowdown of the switching time in the severely fatigued samples. The direct nanoscale observation of domain nucleation and growth dynamics elucidates the importance of evolution of the domain wall pinning process in the fatigue of ferroelectric materials.