In this work, through investigating a series of liquid, glassy, and crystalline samples with ultrafast multiple-mode 2D IR and IR transient absorption methods, we demonstrated that the signal anisotropy of vibrational relaxation-induced heat effects is determined by both relative molecular orientations and molecular rotations. If the relative molecular orientations are randomized or molecular rotations are fast compared to heat transfer, the signal anisotropy of heat effects is zero. If the relative molecular orientations are anisotropic and the molecular rotations are slow, the signal anisotropy of heat effects can be nonzero, which is determined by the relative orientations of the energy source mode and the heat sensor mode within the same molecule and in different molecules. We also demonstrated that the correlation between the anisotropy value of heat signal and the relative molecular orientations can be quantitatively calculated.