Diaryl azo pigments play an important role as yellow pigments for printing inks, with an annual pigment production of more than 50,000 t. The crystal structures of Pigment Yellow 12 (PY12), Pigment Yellow 13 (PY13), Pigment Yellow 14 (PY14), and Pigment Yellow 83 (PY83) were determined from X-ray powder data using lattice energy minimizations and subsequent Rietveld refinements. Details of the lattice energy minimization procedure and of the development of a torsion potential for the biphenyl fragment are given. The Rietveld refinements were carried out using rigid bodies, or constraints. It was also possible to refine all atomic positions individually without any constraint or restraint, even for PY12 having 44 independent non-hydrogen atoms per asymmetric unit. For PY14 (23 independent non-hydrogen atoms), additionally all atomic isotropic temperature factors could be refined individually. PY12 crystallized in a herringbone arrangement with twisted biaryl fragments. PY13 and PY14 formed a layer structure of planar molecules. PY83 showed a herringbone structure with planar molecules. According to quantum mechanical calculations, the twisting of the biaryl fragment results in a lower color strength of the pigments, whereas changes in the substitution pattern have almost no influence on the color strength of a single molecule. Hence, the experimentally observed lower color strength of PY12 in comparison with that of PY13 and PY83 can be explained as a pure packing effect. Further lattice energy calculations explained that the four investigated pigments crystallize in three different structures because these structures are the energetically most favorable ones for each compound. For example, for PY13, PY14, or PY83, a PY12-analogous crystal structure would lead to considerably poorer lattice energies and lower densities. In contrast, lattice energy calculations revealed that PY12 could adopt a PY13-type structure with only slightly poorer energy. This structure was found experimentally as a metastable gamma phase of PY12. Calculations on mixed crystals (solid solutions) showed that mixed crystals of PY12 and PY13 should adopt the PY13 structure with planar molecules, resulting in high color strengths; this was proven experimentally (Pigment Yellow 188). Similarly, the high color strength of mixed crystals consisting of PY13 and PY14 (Pigment Yellow 174), and PY13/PY83 (Pigment Yellow 176) is explained by the crystal structures.