Wind farms operate often in the changing wind. The wind condition variations in a wide range of time scales lead to the variability of wind farms' power production. This imposes a major challenge to the power system operators who are facing a higher and higher penetration level of wind power. Thus, wind farm developers/owners need to take the variability into consideration in the designing/planning stage, in addition to the conventional main objective of maximizing the expected power output under a fixed wind distribution. In this study, we first propose a new metric to evaluate the variability of wind power based on the characteristics of the wind farm and its local wind conditions. Then a series of robustness metrics are proposed to quantify wind farm's ability to produce power with high mean value and low variability under changing wind, considering both short-term and long-term wind condition variations. Based on these metrics, wind farm layout optimization is performed to maximize the robustness of a real offshore wind farm in Denmark. The results demonstrate that the robustness metrics are more flexible and complete than the conventional metrics for characterizing wind farm power production, such as mean power output or wind power variability alone, and it is feasible to design wind farms to produce power with high mean value and low variability. (C) 2017 Elsevier Ltd. All rights reserved.