The optical anisotropy of materials with isotropic bulk crystal structure depends to a large extent on the surface atomic structure. For instance, data obtained by reflectance anisotropy spectroscopy (RAS) on (001) surfaces of zinc blende semiconductors such as InP and GaAs, have a fingerprint character for the various surface reconstructions. Here we present RAS spectra for GaAs(001) and InP(001) recorded at room temperature and at low temperature. We show that by comparison with a theoretical analysis based on ab initio density functional theory in local-density approximation calculations the origin of characteristic spectral features can be identified and thus RAS spectra utilized to discriminate between different competing structural models. We identify contributions related to electronic transitions between surface states as well as features arising from surface perturbed bulk wave functions. We explain the high sensitivity of RAS to the surface structure. and chemistry as due to the surface state related features.