Langmuir-Blodgett (LB) films of poly(5-(2-(4-((4-hexylphenyl)azo)phenoxy)ethyl) L-glutamate) (P2) and poly(5-(6-(4-((4-hexylphenyl)azo)phenoxy)hexyl) L-glutamate) (P6) polymers with a stiff backbone and flexible side chains ("hairy rods"), have been investigated at different steps of the photoisomerization and annealing process by means of X-ray reflection and UV-vis spectroscopy. From the absorption spectra the amount of aggregated chromophores was estimated using a peak-fitting procedure. The original LB structure, which is characterized by deformed "hairy rods’ arranged in bilayers, is destroyed upon photoisomerization or annealing. The side chains relax to a more symmetrical distribution around the main chain helix. Furthermore the aggregation of the chromophores as well as their orientation in respect to the main chain helix is changed. The changes are more pronounced upon irradiation and depend on the length of the spacer used to tether the chromophores to the polymer backbone.