To further reveal the extraction model for selective separation of trivalent actinides over lanthanides by dithiophosphinic acids (DPAHs), five representative DPAH ligands with different substituent groups have been synthesized, and their extraction and complexation behaviors toward Am3+/Eu3+ have been investigated both experimentally and theoretically. The introduction of electron-withdrawing group -CF3 into DPAH ligands is beneficial to their extractability among the five ligands. Slope analyses show that both Am3+ and Eu3+ were extracted as tetra-associated species with DPAH ligands. In addition, the results obtained from luminescence spectroscopy, Raman spectroscopy, and ESI-MS suggest that all of the five DPAHs coordinate with Eu3+ mainly in the form of ML3(HL)-(H2O) (L represents deprotonated DPAH). Density functional theory (DFT) calculations on the thermodynamic parameters illustrate that the extractability of DPAHs is dominated by the deprotonation property of these ligands. Meanwhile, molecular orbital analysis indicates that the unoccupied valence orbitals of Am3+ display a stronger affinity to the sulfur lone electron pair than those of Eu3+, which should be one of the key factors contributing to the excellent selectivity of Am3+ over Eu3+ by DPAH ligands.