We report quantitative calculations of carrier lifetimes in imperfect GaxIn1-xSb/InAs superlattice structures. A microscopic description of imperfections including substitutional anions and interface islands is obtained through a novel strain-dependent empirical pseudopotential calculation. The T matric. of scattering theory is used to take our calculations of scattering lifetimes beyond the Born approximation. including multiple scattering events. Carrier lifetimes are related to the microscopic nature of the defects, their proximity to the interfaces, and the size and shape of interface islands. Anomalous effects due to lattice relaxation are seen to alter hole lifetimes, and their dependence upon position. For isolated isovalent anion defects we predict electron and hole lifetimes as low as 0.2 and 0.8 mu s, respectively, for typical defect concentrations.