The calcium carbonate scale inhibition by two inhibitors, polyacrylic acid (PAA) and 2-phosphonobutane-1,2,4-tricarboxylic acid (PBTCA), has been studied in two heat transfer systems: recirculating cooling water and pool boiling systems. It is found that PBTCA has a better inhibition effect than PAA under identical conditions. The inhibition effect increases with increasing fluid velocity for the cooling water system, whereas in the presence of inhibitors, the fluid velocity has less effect on the scaling behavior. When the initial surface temperature increases, the inhibition efficiency decreases. In the presence of inhibitors, the scaling behavior is insensitive to the change of surface temperature. The relationship between the inhibition effect and the fractal dimension has also been investigated. The results show that the fractal dimension is higher in the presence of inhibitors. The better the inhibition effect, the higher the fractal dimension. XRD and FTIR analyses demonstrate that for the CaCO3 formed in the pool boiling system, the content of vaterite increases with the increase of inhibition effects. The metastable crystal forms of vaterite and aragonite are stabilized kinetically in the presence of inhibitors. The step morphology has been observed by atomic force microscopy. It is shown that the step space on the CaCO3 surface increases in the presence of inhibitors. Moreover, with the increase in inhibition effect, both the step space and the fractal dimension increase. Step bunching is also found and discussed in this paper.