Nucleation and growth of the irradiation-induced void, which is a fatal concern for the thermal conductivity of a structural material, is highly sensitive to the extreme irradiation conditions. A three-dimensional phase field model in conjunction with an explicit nucleation algorithm has been developed and utilized to investigate this issue. The temperature-dependent material parameters of alpha-Fe are derived from ab initio calculations. Phase field simulations reveal the effects of temperature and damage rate on the void formation behavior. At the same temperature, lower damage rate leads to longer incubation time, smaller void radius and less void number, while at the same damage rate, higher temperature causes shorter incubation time, larger void radius but less void number. The effective thermal conductivity of irradiated alpha-Fe is examined and analyzed by taking into account of void volume fraction, which agrees well with the calculated results using the Maxwell and Bauer models. The void formation behavior revealed by the phase field simulations and the effect of void volume fraction on thermal conductivity could be helpful in understanding the irradiation damage of structural materials.