When subject to flow, entangled polymer melts and wormlike micellar solutions can form adjacent regions having a shear-rate discontinuity at their interface. This phenomenon is known as shear banding, and has recently been observed using NMR microscopy [M.M. Britton, P.T. Callaghan, Phys. Rev. Lett. 78 (1997) 4930]. However, the mechanisms involved in the banding process are not well understood. One important mechanism may involve secondary flow, and in this work we study the effects of shear banding on secondary flow in viscoelastic fluids between a cone and plate. We model each band as an Oldroyd-B fluid and apply regular perturbation theory assuming that the cone angle is small. Flows with both two and three bands are considered, and we study the effects of elasticity and viscosity contrast between the bands. Elasticity contrast is found to mainly affect the strength of the secondary flow, and can even cause its direction of circulation to reverse. Viscosity contrast is found to localize the secondary flow in the less viscous band, in addition to affecting its strength. These results may help explain why the experimentally observed shear banded structure is relatively stable. They also suggest an interesting mechanism for the formation of cusps at free surfaces.