With increasing levels of integration of intermittent renewable energy in the power grid, it has become essential for power system planners to quantify future requirements of power systems in terms of flexibility. It is also equally important to evaluate whether the flexibility available in a given power system is adequate to meet more frequent and larger variations in the net load. In this paper, we present a novel framework to develop a composite metric that provides an accurate assessment of flexibility within conventional generators of a power system. This assessment is performed using eight technical characteristics of generating units as indicators. An Analytic Hierarchy Process is applied to assign weights to these indicators in order to reflect their relative importance in the supply of flexibility. Following normalization with min-max method, the indicators are linearly aggregated to give the composite flexibility index for each generator. The proposed methodology is tested on an adapted IEEE RTS-96 system. Our results demonstrate the consistency of the composite flexibility metric. It is further observed that the proposed metric is adaptive since it automatically adjusts to the addition and/or removal of generating units. To evaluate the robustness of the proposed framework, we also performed sensitivity and uncertainty analysis in the presence of alternative methodological choices in the composite metric construction process. (C) 2016 Elsevier Ltd. All rights reserved.