The free energies of clathrates are calculated over a wide range of temperatures in order to explain a large thermal expansivity of clathrate hydrates compared with that of ice. Several proton-disordered configurations for hexagonal ice and type I chathrate hydrates are generated. The free energy is approximated to the sum of the minimum potential energy, the harmonic free energy, and the configurational entropy arising from the disordered nature of protons. The free energy at a given temperature is minimized with respect to the volume of the system. This enables us to evaluate the thermal expansivity from only intermolecular interaction potentials. The larger thermal expansivity of clathrate hydrates than ice is successfully reproduced. It is found that the large expansivity of clathrate hydrate structure I stems from the existence of guest molecules and that a difference in oxygen atom arrangement between clathrate hydrates and ice plays a minor role. The effective potential energy surface of a guest molecule becomes harmonic with an increase in temperature. This seems to undermine the large difference in the thermal expansivity between clathrate hydrates and ice.