This paper reports the evaluation of dispersion models with respect to their ability to predict the dispersion of continuous dense gas plumes over flat terrain. The exercise has been carried out in order to derive guidelines for statistical model evaluation exercises. Therefore, the models selected for the evaluation are simple: the Britter and McQuaid model and three variations of the Gaussian Plume Model. Two different evaluation procedures have been used: one using short time-averaged concentrations paired in time and space and another using observed maximum concentrations at certain distances together with a measure of the plume width. The performance numbers and ranking of the models depend substantially on the data sets used and the evaluation procedure and performance number selected. The two evaluation procedures have both advantages and disadvantages. An evaluation based on maximum concentration should not be performed without taking into account the plume dimensions. The evaluation using data paired in time and space suffers from the inability of the models and the data analysis techniques to match the predicted plume path with the actual plume path during the experiment. We suggest that two newly defined performance measures be used, the Mean Relative Bias and the Mean Relative Square Error, together with the well-known ＇Factor-of-2＇ statistic in future statistical evaluation exercises. One should use at least two performance measures in an evaluation exercise in order to obtain a measure of ＇bias＇ and ＇variance＇. Furthermore, we advise that graphical presentations to obtain an understanding of the model behaviour be used.