The surface roughening of films deposited atomically and eroded by ion irradiation sputtering, resulting from the stochastic or noisy natures of these processes, is investigated analytically. Serial modes of successive deposition and irradiation and parallel modes of congruent deposition and irradiation are compared and the superiority of one or the other mode shown to depend upon growth and erosion-rate ratios. Different surface relaxation processes including viscous flow and surface diffusion are examined and shown to affect both the average surface roughness and the spatial frequency spectrum of the roughness.