This work presents a detailed, combined experimental and theoretical study on the structural stability of s-p bonded compounds with the BaAl4 structure type (space group /4/mmm, Z = 2) as part of a broad program to investigate the complex questions of structure formation and atomic arrangements in polar intermetallics. From ab initio calculations employing pseudopotentials and a plane wave basis set, we extracted optimized structural parameters, binding energies, and the electronic structure of the systems AeX(III)(4), AeX(II)(2)X(IV)(2), AeX(II)(2)X(III)(2) (Ae = Ca, Sr, Ba; X(II) = Mg, Zn; X(III) = Al, Ga; X(IV) = Si, Ge). For all systems we found a pronounced pseudo-gap in the density of states separating network X-4(2-) bonding from antibonding electronic states that coincides with the Fermi level for an electron count of 14 electrons per formula unit, the optimum value for stable BaAl4-type polar intermetallics. However, the synthesis and structural characterization (from X-ray single crystal and powder diffraction data) of the new compounds AeZn(2-delta)Al(2+delta), AeZn(2-delta)Ga(2+delta) (Ae = Ca, Sr, Ba; delta = 0-0.2) and AeMg(0.9)Al(3.1), AeMg(1.7)Ga(2.3) (Ae = Sr, Ba) manifested that electron deficiency is rather frequent for BaAl4-type polar intermetallics. The site preference for different "X" elements in the ternary systems was quantified by calculating "coloring energies", which, for some systems, was strongly dependent on the size of the electropositive Ae component. The Ae(2+) cations decisively influence the nearest neighbor distances in the encapsulating polyanionic networks X-4(2-) and the structures of these networks are surprisingly flexible to the size of the Ae component without changing the overall bonding picture. A monoclinically distorted variant of the BaAl4 structure occurs when the cations become too small for matching the size of encapsulating X-4(2-) cages. An even larger size mismatch leads to the formation of the Euln(4) structure type.