To simultaneously achieve low photon energy loss (E-loss) and broad spectral response, the molecular design of the wide band gap (WBG) donor polymer with a deep HOMO level is of critical importance in fullerene-free polymer solar cells (PSCs). Herein, we developed a new benzodithiophene unit, i.e., DTBDT-EF, and conducted systematic investigations on a WBG DTBDT-EF-based donor polymer, namely, PDTB-EF-T. Due to the synergistic electron-withdrawing effect of the fluorine atom and ester group, PDTB-EFT exhibits a higher oxidation potential, i.e., a deeper HOMO level (ca. -5.5 eV) than most well-known donor polymers. Hence, a high open-circuit voltage of 0.90 V was obtained when paired with a fluorinated small molecule acceptor (IT-4F), corresponding to a low E-loss, of 0.62 eV. Furthermore, side-chain engineering demonstrated that subtle side-chain modulation of the ester greatly influences the aggregation effects and molecular packing of polymer PDTB-EF-T. With the benefits of the stronger interchain pi-pi interaction, the improved ordering structure, and thus the highest hole mobility, the most symmetric charge transport and reduced recombination are achieved for the linear decyl-substituted PDTB-EF-T (P2)-based PSCs, leading to the highest short-circuit current density and fill factor (FF). Due to the high Flory-Huggins interaction parameter (chi), surface-directed phase separation occurs in the P2:IT-4F blend, which is supported by X-ray photoemission spectroscopy results and cross-sectional transmission electron microscope images. By taking advantage of the vertical phase distribution of the P2:IT4F blend, a high power conversion efficiency (PCE) of 14.2% with an outstanding FF of 0.76 was recorded for inverted devices. These results demonstrate the great potential of the DTBDT-EF unit for future organic photovoltaic applications.