Multidimensional vibrational response functions of a harmonic oscillator are reconsidered by assuming nonlinear system-bath couplings. In addition to a standard linear-linear (LL) system-bath interaction, we consider a square-linear (SL) interaction. The LL interaction causes the vibrational energy relaxation, while the SL interaction is mainly responsible for the vibrational phase relaxation. The dynamics of the relevant system are investigated by the numerical integration of the Gaussian-Markovian Fokker-Planck equation under the condition of strong couplings with a colored noise bath, where the conventional perturbative approach cannot be applied. The response functions for the fifth-order nonresonant Raman and the third-order infrared (or equivalently the second-order infrared and the seventh-order nonresonant Raman) spectra are calculated under the various combinations of the LL and the SL coupling strengths. Calculated two-dimensional response functions demonstrate that those spectroscopic techniques are very sensitive to the mechanism of the system-bath couplings and the correlation time of the bath fluctuation. We discuss the primary optical transition pathways involved to elucidate the corresponding spectroscopic features and to relate them to the microscopic sources of the vibrational nonlinearity induced by the system-bath interactions. Optical pathways for the fifth-order Raman spectroscopies from an "anisotropic" medium were newly found in this study, which were not predicted by the weak system-bath coupling theory or the standard Brownian harmonic oscillator model. (C) 2004 American Institute of Physics.