Silver nanoparticles (4.5 nm average radius) are used as the first negatively charged layer of a polyelectrolyte multilayer prepared subsequently from poly(allylamine hydrochloride) (PAH) and poly(styrenesulfonate) (PSS) solutions with 1 M KCl. After adsorption of a PAH layer on top of the silver nanoparticle monolayer, particle aggregation occurs, as evidenced by the double peak of the UV-vis absorption spectrum and by the decreased number of the objects found with AFM. On adsorption of the first PSS layer on top of the Ag/PAH sandwich, the nanoparticles remain close as is indicated by a the reduced object count and the changed absorption spectrum. If the PAH layer covering the Ag nanoparticles is adsorbed from salt-free solution, the nanoparticles remain isolated. Apparently, the aggregation is mediated by the PAH adsorbing in coiled conformation. Additionally, UV-vis and X-ray reflectivity evidence is found for lateral yet not vertical nanoparticle movement when polyelectrolytes adsorb, even if the adsorbing coils do not touch the nanoparticles directly.