Vibrational spectra of several isotopomers of benzoic acid (BA) crystals have been recorded by inelastic neutron scattering and are compared with spectra calculated for different potential energy surfaces (PES). These PES were obtained within the harmonic approximation from quantum chemical density functional theory (DFT) calculations made for the monomer, the isolated dimer, and the crystal using different codes and different levels of basis functions. Without refinement of the force constants, agreement between calculated and observed spectra is already sufficient for an unambiguous assignment of all vibrational modes. The best agreement was obtained with periodic DFT calculations. The most prominent discrepancy between calculated and observed frequencies was found for the out-of-plane O-H bending modes. For these modes (as well as for the in-plane bending and the O-H stretching modes) the anharmonicity of the potential was calculated, and the anharmonic correction was shown to account for about one-third of the discrepancy. The origin of this difference is attributed to the slight compression of the hydrogen bonds in the calculated structure of the dimer, which also leads to a significant lowering of the frequency of the O-H stretch mode.