The internal structure of composite gels made of responsive microgel particles inserted into a bulk hydrogel (N-isopropylacrylamide microgel particles in a cross-linked dimethylacrylamide matrix) has been investigated from the diffusion behavior of poly(ethylene glycol) (PEG) probes through the network, in the absence of specific interactions between the diffusing molecules and the system. The effect of the different components has been examined, for example, the size of the probe, the bulk structure, and the microgel nature. Particles were characterized prior to their insertion into the hydrogel in order to describe their properties as a function of size and cross-linker content, thus revealing different swelling behaviors. The biggest effects on the diffusion of the PEG probes were related to the bulk structure, and no major effects were registered by the addition of different microgels into the hydrogel network. We attempt to rationalize this behavior in terms of the composite gel structure and discuss the results in terms of their meaning for controlled drug delivery strategies.