We study the appearance and energy of the charge transfer (CT) state using measurements of electroluminescence (EL) and photoluminescence (PL) in blend films of high-performance polymers with fullerene acceptors. EL spectroscopy provides a direct probe of the energy of the interfacial states without the need to rely on the LUMO and HOMO energies as estimated in pristine materials. For each polymer, we use different fullerenes with varying LUMO levels as electron acceptors, in order to vary the energy of the CT state relative to the blend with [6,6]-phenyl C61-butyric acid methyl ester (PCBM). As the energy of the CT state emission approaches the absorption onset of the blend component with the smaller optical bandgap, E-opt,E-min equivalent to min{E-opt,E-donor; E-opt,E-acceptor}, we observe a transition in the EL spectrum from CT emission to singlet emission from the component with the smaller bandgap. The appearance of component singlet emission coincides with reduced photocurrent and fill factor. We conclude that the open circuit voltage V-OC is limited by the smaller bandgap of the two blend components. From the losses of the studied materials, we derive an empirical limit for the open circuit voltage: V-OC less than or similar to E-opt,E-min/e - (0.66 +/- 0.08)eV.