Lithium-ion conducting argyrodites have recently attracted significant interest as solid electrolytes for solid-state battery applications. In order to enhance the utility of materials in this class, a deeper understanding of the fundamental structure-property relationships is still required. Using Rietveld refinements of X-ray diffraction data and pair distribution function analysis of neutron diffraction data, coupled with electrochemical impedance spectroscopy and speed of sound measurements, the structure and transport properties within Li6PS5-xSexBr (0 <= x <= 1) have been monitored with increasing Se content. While it has been previously suggested that the incorporation of larger, more polarizable anions within the argyrodite lattice should lead to enhancements in the ionic conductivity, the Li6PS5-xSexBr transport behavior was found to be largely unaffected by the incorporation of Se2- due to significant structural modifications to the anion sublattice. This work affirms the notion that, when optimizing the ionic conductivity of solid ion conductors, local structural influences cannot be ignored and the idea of "the softer the lattice, the better" does not always hold true.