Transmission microscopy with soft X-rays (TXM) is applied to image in-situ polyelectrolyte assemblies in aqueous environment. The method is element specific and at this stage exhibits a lateral resolution of 20 nm. With the specific examples of hollow capsules and full spheres made of PAH/PSS polyelectrolyte multilayers, it is shown quantitatively that heat treatment irreversibly reduces the water content in the membrane. These experiments complement those reported recently on the polyion system PDADMAC/PSS, which shows a different glass-transition behavior. Finally, the potential and present limitations of TXM are discussed.