The MBE-grown GaAs(001) surface exhibits various surface phases depending on the surface temperature and As/Ga flux ratio during the growth. Using scanning tunneling microscopy, reflection high energy electron diffraction and theoretical simulations, we have carried out a systematic study of various phases from As-rich c(4 X 4), 2 X 4 to Ga-rich 4 X 2 and 4 X 6, utilizing migration-enhanced epitaxy. Based on our thorough investigation, we were able to propose a simple and unified structural model for the evolution of surface phases. The As-rich 2 X 4 phase consists of two As dimers on the top layer and another As dimer on the third layer (Chadi’s two-dimer model), while there is still a small window where a slightly As-poor phase may exist consisting of two As dimers on the top layer and second layer relaxation (Northrup Froyen model). As for the Ga-rich 4 X 2, we determined that the Beigelsen’s two-Ga-dimer model fits best to our experimental and theoretical results, which consists of two Ga dimers on the top layer and an additional Ga dimer on the third layer, a minor image of Chadi’s two-dimer model for the As-rich 2 X 4 phase. Our direct observations reveal that there are two distinct 4 X 6 phases.