Aimed at the determination of the number of mixtures for finite mixture models (FMMs), in this work, a new method called the penalized histogram difference criterion (PHDC) is proposed and evaluated with other criteria such as Akaike information criterion (AIC), the minimum message length (MML), the information complexity (ICOMP) and the evidence of data criterion (EDC). The new method, which calculates the penalized histogram difference between the data generated from estimated FMMs and those for modeling purpose, turns out to be better than others for data with complicate mixtures patterns. It is demonstrated in this work that the PHDC can determine the optimal number of clusters of the FMM. Furthermore, the estimated FMMs asymptotically approximate the true model. The utility of the new method is demonstrated through synthetic data sets analysis and the batch-wise comparison of citric acid fermentation processes. (C) 2013 Elsevier Ltd. All rights reserved.