We consider the linear stability of shear banded planar Couette flow of the Johnson-Segalman fluid, with and without the addition of stress diffusion to regularise the equations. In particular, we investigate the linear stability of an initially one-dimensional "base" flow, with a flat interface between the bands, to two-dimensional perturbations representing undulations along the interface. We demonstrate analytically that, for the linear stability problem, the limit in which diffusion tends to zero is mathematically equivalent to a pure (non-diffusive) Johnson-Segalman model with a material interface between the shear bands, provided the wavelength of perturbations being considered is long relative to the (short) diffusion lengthscale. For no diffusion, we find that the flow is unstable to long waves for almost all arrangements of the two shear bands. In particular, for any set of fluid parameters and shear stress there is some arrangement of shear bands that shows this instability. Typically the stable arrangements of bands are those in which one of the two bands is very thin. Weak diffusion provides a small stabilising effect, rendering extremely long waves marginally stable. However, the basic long-wave instability mechanism is not affected by this, and where there would be instability as wavenumber k -> 0 in the absence of diffusion, we observe instability for moderate to long waves even with diffusion. This paper is the first full analytical investigation into an instability first documented in the numerical study of Fielding [S.M. Fielding, Linear instability of planar shear banded flow, Phys. Rev. Lett. 95 (2005) 134501]. Authors prior to that work have either happened to choose parameters where long waves are stable or used slightly different constitutive equations and Poiseuille flow, for which the parameters for instability appear to be much more restricted. We identify two driving terms that can cause instability: one, a jump in N-1, as reported previously by Hinch et al. [E.J. Hinch, O.J. Harris, J.M. Rallison, The instability mechanism for two elastic liquids being coextruded, J. Non-Newtonian Fluid Mech. 43 (1992) 311-324]; the second, a discontinuity in shear rate. The mechanism for instability from the second of these is not thoroughly understood. We discuss the relevance of this work to recent experimental observations of complex dynamics seen in shear-banded flows. (c) 2006 Elsevier B.V. All rights reserved.