Fluorescence lifetime measurements have been performed on single tetraphenoxy-perylenetetracarboxy diimide molecules embedded in two different polymer matrices. Single molecule decay times are found to fluctuate in time as a result of the dye molecule-polymer chains interaction, causing conformational rearrangements of the dye molecule. The rearrangements are shown to take place on a 2 ms time scale at some locations of the dye molecule within the polymer matrix. This behavior illustrates the spatially heterogeneous dynamics present in polymers at room temperature and provides a quantified estimation of the time scales involved in the process.