A conjugate gradient method based inverse algorithm is applied in the present study in simultaneous determining the unknown time-dependent Biot numbers of heat and moisture transfer for a porous material based on interior measurements of temperature and moisture. It is assumed that no prior information is available on the functional form of the unknown Biot numbers in the present study, thus, it is classified as the function estimation in inverse calculation. The accuracy of this inverse heat and moisture transfer problem is examined by using the simulated exact and inexact temperature and moisture measurements in the numerical experiments. Results show that the estimation on the time-dependent Biot numbers can be obtained with any arbitrary initial guesses on a Pentium IV 1.4 GHz personal computer.