Despite being intensively studied, the magnitude of specific structural and dynamic perturbations of water next to hydrophobic surfaces remains a matter of debate. Here we show, from molecular dynamics, that the structure of a subset of water molecules in the first hydration layer, those preserving four nearest water neighbors, resembles that of water at 10 degrees C, and that the origin of the orientational slowdown is mainly a decrease of the hydrogen-bond (HB) acceptor switch frequency, while water structuring plays a minor role, slightly accelerating HB acceptor switches. By portraying the mean HB dynamics of water as a doubly periodic event, we demonstrate that the orientational retardation factor is effectively defined by the ratio of the HB acceptor switch period in the hydration layer and bulk. Excluded volume delays HB acceptor switches, accelerating the orientational relaxation of similar to 1/3 of the water molecules on the hydration layer in this time scale, but this is largely exceeded by the decrease of the HB switch frequency, consistent with 2D IR spectroscopy experiments, and at the origin of longer HB lifetimes. The orientational mobility of water populations with long HB lifetimes is also probed, and although a relaxation plateau is observed at similar to 10 ps consistent with fs IR spectroscopy experiments, no water molecule is rotationally frozen at any time scale. The proposed molecular picture is consistent with fs IR, 2D IR, and NMR experimental results on the orientational retardation of water and reveals the magnitude of "hidden" enhanced ordered water pentamers formed near hydrophobic solutes.