We demonstrate azobenzene photochemically driven diffraction switching of a photonic crystal consisting of a crystalline colloidal array (CCA) polymerized within a hydrogel matrix. A novel azobenzene derivative that has a large ground-state activation barrier between the cis and trans forms in water is used. The system is actuated by excitation with UV light (wavelength of 365 nm), which photoisomerizes the azobenzene trans state to the cis ground state. The increased dipole moment of the cis state increases the free energy of mixing, causing a hydrogel swelling, which red-shifts the embedded CCA diffraction. Excitation with visible light photoisomerizes the cis state to the trans state, which resets the diffraction. This material acts as a memory storage material. Information is recorded and erased by exciting the photonic crystal in the UV or visible spectral region. The written information is read out completely and nondestructively by the wavelength of the Bragg diffraction (in this case, in the red).