We introduce the time-resolved phosphorescence anisotropy (TPA) method to study rotational diffusion of phosphorescent colloidal silica spheres in dispersions and in random close packings of host spheres. The tracer diffusion coefficients in sphere dispersions agree with hard-sphere predictions, demonstrating that TPA is a reliable technique for studying rotational dynamics. We also assess the rotational diffusion of phosphorescent tracers in sphere packings, exploiting the fact that the TPA technique can be used for slightly turbid samples. We find that the fraction of immobilized tracers directly reflects the pore size distribution as extracted from the Voronoi construction of a simulated random close packing. A calculation of the average distance between tracer and neighboring particle shows that the dependence of the rotational diffusion coefficient on the average distance between tracer and medium spheres is stronger for packings than for dispersions.