In this work, we combine Bayesian techniques with a color categorization model, which leads to a method for the linguistic segmentation of color images. The categorization model considers the 11 universal color categories proposed by Berlin and Kay [Basic Color Terms: Their Universality and Evolution. Berkeley: University of California; 1969]. The likelihood for each category, is represented by a linear combination of quadratic splines, and as a result, each voxel in the color space L*u*v* is described as a vector of probabilities, whose components express the degree to which the voxel belongs to a given color category. This gives rise to a probabilistic dictionary which is used for the segmentation, in which prior spatial granularity constraints are incorporated via an entropy-controlled quadratic Markov measure field (ECQMMF) model, as proposed by Rivera et al. [IEEE Trans Image Process 2007;16:3047-3057]. We give a generalization of ECQMMF that allows one to consider the perceptual interactions between the basic colors that were experimentally established by Boynton and Olson [Color Res Appl 1987;12.-94-105]. (C) 2009 Wiley Periodicals. Inc. Col Res Appl. 34. 299-309. 2009: Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/col.20509