Supercritical CO2 Rankine cycle has great development potential as a power cycle for converting low-grade thermal energy into electricity. Better understanding of supercritical CO2 heat transfer in helically coiled tubes (HCTs) is required for design and operation of supercritical CO2 Rankine cycle power systems. In this work, the SST k similar to omega model is employed. A new dimensionless buoyancy parameter VI is proposed which denotes the ratio of gravitational buoyancy force to overall curvature effect. Furthermore, a flow regimes map is proposed based on the inclination angle of the dividing streamline between the two vortexes and buoyancy parameter Psi. The mixed convection region in HO' is decomposed into a gravitational buoyancy force dominated heat transfer region (B Region alpha > 45 degrees) and a curvature effect dominated heat transfer region (C Region alpha <45 degrees). Subsequently, the effects of HCT geometry on heat transfer mechanisms are respectively investigated in B and C Region, which help us better understanding the relationship of the buoyancy criterion and flow characteristics. The results indicate that the effects of coiled pitch and coiled diameter on heat transfer can be neglected in B Region. In C Region, the heat transfer is suppressed as coiled pitch increases and it will appear oscillation when torsion effect is strong enough. In addition, the heat transfer is enhanced with curvature increases but except for near the pseudo-critical region. (C) 2019 Published by Elsevier Ltd.