We investigate the effect of glassy rubbery interfaces between neighboring polymer domains on the local stability and physical aging of confined glassy layers. We present results demonstrating how ellipsometry can be used to measure the physical aging rate of glassy polystyrene (PS) layers atop rubbery poly(n-butyl methacrylate) (PnBMA) layers. With decreasing PS layer thickness, down to 84 nm, we observe no change in PS aging rate despite a 25-30 degrees C reduction in the average glass transition temperature T-g of this layer. These results are in strong contrast to single layer PS films, which show a decrease in aging rate with decreasing thickness due to the local T-g reduction at the free surface. The aging rate of the glassy PS layers atop rubbery PnBMA cannot be explained by any shift in local T-g, suggesting some additional factor is affecting the structural relaxation occurring near the glassy rubbery interface.