Photoelectron detachment measurements have been performed on singly charged phosphorus cluster anions with up to nine atoms, generated by a pulsed are cluster ion source (PACIS). Transitions between the anion ground states and states of the neutral clusters are observed for all clusters, and vibrational fine structure in both dimer and trimer. A comparison with the results of density functional calculations with simulated annealing - an extension to negative ions of earlier work on neutral and positively charged clusters - provides a consistent overall picture for all cluster sizes and the first experimental structural information on several.