Small-angle neutron scattering experiments show that the structure of the hydrogels formed by the guanosine hydrazide 1 is characterized by a network of thick fibers, whereas the gels obtained by condensation of 1 with pyridoxal monophosphate 2 result from the formation of single semi-flexible columns of self-assembled quartets. The observed structural differences might explain the different rheological behaviors exhibited by these gels. They point to the ability to modulate the supramolecular structure of hydrogel assemblies by dynamic covalent decoration of the core entity.