Extraordinary oxygen ordering and mobility have been found in an oxygen-nonstoichiometric mixed-valence cobalt oxide, YBaCo4O8.5. The excess oxygen atoms appear to be incorporated in an orderly way into the YBaCo4O7 parent lattice with different configurations as the oxygen content varies. Intense electron-beam irradiation was used to observe recurrent oxygen migration in the lattice, causing reversible structural modulation transitions. The oxygen migration can be attributed mainly to the electron-beam heating effect. This study highlights the high degree of freedom of surplus oxygen within the investigated structure and advances our understanding of the oxygen diffusion process in related transition-metal oxide systems.