Manganese oxides are good candidates of strongly correlated electron materials due to the uniqueness of electronic structure of manganese and the mobility of oxygen among lattice sites under external impacts. Here, we used electron beam as the excitation source to explore the structural evolution of YMnO3 and identified a new phase under the radiation of electron beam in the transmission electron microscope. Analyses of the electron energy-loss spectra reveal that this phase originates from ordered oxygen vacancy. We applied the first principles calculation to pick out the optimized stable structure with a lower polarization, and verified its correctness by electron diffraction and image simulations. Analyses of density of states indicate that weak Y-O covalence is favorable for the existence of ferroelectricity, supporting the electrostatic nature of ferroelectricity in the YMnO3.