Random terpolymers with three different monomer units can provide broader light absorption than the most widely used donor-acceptor (D-A) alternating copolymers, but their electrical properties are often sacrificed by the randomly distributed monomers in the polymeric backbone that prevent efficient intermolecular pi-pi interactions. Here, we report the development of a regioregular terpolymer and demonstrate its importance in enhancing the power conversion efficiency (PCE) of all-polymer solar cells (all-PSCs). To investigate the impact of the monomer sequence and regioregularity in the terpolymer, we designed and synthesized two terpolymers (Ra-(D-1-A-D-2-A) random terpolymer and RR-(D-1-A-D-2-A) regioregular terpolymer) consisting of two electron-donating benzodithiophene (BDT) units with different side chains and one electron-withdrawing fluorinated thieno[3,4-b]thiophene (TT-F) unit. As a reference polymer, we also synthesized the D-1-A alternating copolymer. The RR-(D-1-A-D-2-A) film exhibited stronger pi-pi stacking and a larger crystallite size than the D-1-A and Ra-(D-1-A-D-2-A) films, resulting in 1 order of magnitude higher hole mobility than those of the other polymers. When blended with the P(NDI2HD-DTAN) polymer acceptor, the RR-(D-1-A-D-2-A)-based all-PSC yielded an outstanding PCE of 6.13%, which was superior to those of the D-1-A-based all-PSCs (4.81%) and Ra-(D-1-A-D-2-A)-based all-PSCs (4.93%). These findings indicate that the synthesis of the regioregular terpolymer is a promising design strategy for the development of high-performance all-PSCs with improved optical and electrical properties.