The linear viscoelastic response of a suspension of Brownian hard spheres without hydrodynamic interaction is studied in an approximation in which the influence of neighboring particles on the dynamics of a selected pair is taken into account via the potential of mean force. It is shown that the cage effect due to the barrier imposed by neighbors has a profound influence on the frequency-dependence of the viscoelastic response. Analytic calculations are performed for a square well-barrier model of the potential of mean force.