Layout optimization is one of the methods to increase wind farm's utilization rate and power output. Previous researches have revealed that different hub height wind turbines may increase wind farm's power output. However, few researches focus on optimizing a wind farm's layout in a two-dimensional area using different hub height wind turbines. In this paper, the authors first investigate the effect of using different hub height wind turbines in a small wind farm on power output. Three different wind conditions are analyzed using nested genetic algorithm, where the results show that power output of the wind farm using different hub height wind turbines will be increased even when the total numbers of wind turbines are same. Different cost models are also taken into account in the analysis, and results show that different hub height wind turbines can also improve cost per unit power of a wind farm. At last, a large wind farm with commercial wind turbines is analyzed to further examine the benefits of using different hub height wind turbines in more realistic conditions. Published by Elsevier Ltd.