Nitrogen atoms in the cleaved (110) surfaces of dilute GaAsN and InGaAsN alloys have been studied using cross-sectional scanning tunneling microscopy. The distribution of nitrogen atoms in GaAs0.983N0.017 and In0.04Ga0.96As0.09N0.01 alloys is found to be in agreement with random statistics, with the exception of a small enhancement in the number of [001]-oriented nearest-neighbor pairs. The effects of annealing on In0.04Ga0.96As0.99N0.01 alloys has been studied by scanning tunneling spectroscopy. Spectra display a reduced band gap compared to GaAs but little difference is seen between as-grown versus annealed InGaAsN samples. In addition, voltage dependent imaging has been used to investigate second-plane nitrogen atoms.