The intentional addition of small amounts of nitrogen to different C/H/O gas systems in microwave plasma-assisted deposition of diamond films at low substrate temperatures has been studied. The effect on growth is qualitatively different for gas mixtures with or without oxygen. Adding nitrogen to C/H mixtures results in a significant change of film morphology, growth rate, defect formation and incorporation of hydrogen. The film quality seriously deteriorates with increasing nitrogen concentration in the gas phase. The influence of nitrogen on gas phase processes has been monitored by optical emission spectroscopy. There is evidence that nitrogen affects growth primarily by surface related mechanisms. By contrast, its effect on growth from CO-rich C/H/O systems is much less pronounced. These films show a constant quality and a lower defect content. The interaction of nitrogen and oxygen in low temperature growth of diamond films has been thoroughly examined for gas mixtures containing comparatively low oxygen fractions. The presence of oxygen effectively counteracts the deleterious effect of nitrogen on the formation of defects. Elastic recoil detection has shown, however, that the incorporation of nitrogen into the film always increases when its gas phase concentration is raised, no matter which gas system is chosen.