We investigate the temperature dependency of the elastic constants of polyelectrolyte multilayers made from poly(diallyldimethylammonium chloride) (PDADMAC) and poly(styrenesulfonate) (PSS) by measuring the stiffness of individual hollow polyelectrolyte multilayer capsules in water using AFM force spectroscopy. Statistical analysis of the deformation data of the capsule ensemble combined with continuum mechanical modeling allows quantifying changes in the deformation characteristics and respective changes in the Young's modulus of the wall material. Our results show that the Young's modulus of the wall material decreases from the regime of 100 MPa to the order of MPa above 35 degrees C. This transition is reversible when returning to room temperature. The modulus becomes dependent on the deformation rate for high temperature, while it is not rate-dependent for low temperature. Therefore, we conclude that the wall material undergoes a melting process from a glassy to a viscoelastic fluid state. At the same time, a shrinking of the capsules is observed, which we explain qualitatively with surface tension effects. We discuss the implications of this finding in comparison with other multilayer systems and discuss novel strategies for shape control in PE multilayer systems based on these effects.