We investigate the effect of point defects on the dynamics of acoustic and optical phonons by means of optical reflectivity and Raman measurements. Introduction of the defects by means of ion irradiation enhances the dephasing of phonons. The linear increase in the dephasing rate of the coherent acoustic phonon of graphite is explained quantitatively by assuming the scattering of the propagating acoustic phonon by single-vacancies. The dephasing rates of the optical phonons of graphite and GaAs are also linearly increased with increasing ion fluence, which can be described by the mass-defect scattering model. (C) 2002 Elsevier Science B.V. All rights reserved.