Ultrafast infrared (IR) Raman spectroscopy was used to measure vibrational energy transfer between nitrobenzene nitro and phenyl groups, in the liquid state at ambient temperature, when ortho substituents (-CH3, F) were introduced. Quantum chemical calculations were used to assign the vibrations of these molecules to three classes, phenyl, nitro, or global. Combining transient anti-Stokes and Stokes Raman spectra determined the energies of multiple molecular vibrational modes, which were summed to determine the aggregate energies in the phenyl, nitro, or global modes. In a previous study (Pein, B. C.; Sun, Y.; Dlott, D. D., J. Phys. Chem. A 2013, 117, 6066-6072) it was shown that, in nitrobenzene, there was no energy transfer from nitro to phenyl or from nitro to global modes, but there was some transfer from phenyl to nitro and phenyl to global. The ortho substituents activated energy flow from nitro-to-phenyl and nitro-to-global and reduced phenyl-to-nitro flow. The -CH3 substituent entirely shut down the phenyl-to-nitro pathway, presumably by efficiently directing some of the phenyl energy into methyl bending excitations. There is (inefficient) unidirectional vibrational energy flow in nitrobenzene only in the nitro-to-phenyl direction, whereas in o-nitrotoluene, vibrational energy flows only in the nitro-to-phenyl direction.