The pH-controlled assembly of gold nanoparticles within a polymer-gold nanoparticle (AuNP) composite material is described. Poly(allylamine) (PAAm) is used as a reducing and stabilizing agent to generate a stable water-soluble polymer-AuNPs composite. The optical properties and the morphology of the composite material are sensitive to the solution pH due to structural changes of the polymer. As a result, the nanoparticles undergo reversible assembly/dispersion processes controlled by the solution pH. The role of the PAAm in the formation of the assemblies is confirmed by infrared spectroscopy and place exchange reactions.