Holographic storage is one of the most important applications in the field of optics, especially for recording and retrieving data, and information storage by interference patterns in photosensitive materials are no exception in this regard. In this work, we give evidence that holograms recorded by interference of two coherent laser beams in azo dye doped polymer films can be controlled by a third incoherent assisting laser beam. We show that light diffraction can be increased or decreased by an assisting beam depending on the respective orientation of the polarizations of the recording and the assisting beams. We also found that photomanipulation of polarization holograms, prepared by polarization modulation, does not depend on the polarization of the assisting beam, whereas, photomanipulation of holograms prepared by intensity modulation strongly depends on the polarization of the assisting beam. Photoselection is shown to play a major role in the photoassisted diffraction process.