The temperature dependence of infrared absorption spectra of CH and CD on diamond nanocrystal surfaces has been investigated. Phase relaxation was closely examined by analyzing frequency shifts and line broadening in the spectra. Based on the model of Persson and Ryberg [Phys. Rev. B 40, 10 273 (1989)], coupling phonons responsible for the pure dephasing process were found to resonate at omega(0) approximate to 1200 cm(-1) for the CH stretch. By including both the phase and energy relaxation in the linewidth analysis and assuming that energy relaxes via three-phonon emission, we estimate a pure dephasing time of T-2* approximate to 340 ps at room temperature. This value is one order of magnitude larger than the energy relaxation time, T-1 approximate to 19 ps, measured by Chin et al. [Europhys. Lett. 30, 399 (1995)] on a C(111) single crystal surface. We interpret the anomalous observation to be the result of the high frequency of the coupling phonons. For the CD stretches, however, severe line broadening due to exceedingly rapid energy relaxation disallows accurate determination of T-2*. Since CH and CD stretches have similar thermal and inhomogeneous broadening characteristics, the lifetime of the latter is estimated to be T-1 approximate to 220 fs. This T-1 value is one of the shortest vibrational energy relaxation times for any adsorbate on a dielectric surface. Systematic comparison of these anomalous results with the measurements of SiH stretches on Si(100) and Si(111) is given.