We study the optical response of Frenkel excitons in molecular J aggregates with a cylindrical geometry. Such aggregates have recently been prepared for a class of cyanine dyes and are akin to the rod- and ring-shaped light-harvesting systems found in certain bacteria. The linear absorption spectrum exhibits two lines with perpendicular polarization that are separated by a "ring energy scale", which is set by the circumference of the cylinder and the intermolecular transfer interaction in the circumferential direction. On the other hand, the pump-probe spectrum shows bleaching and induced absorption features that are separated by a much smaller energy scale, which is determined by an effective Pauli gap imposed by the length of the cylinder and the transfer interaction in its longitudinal direction. We show that this can be well-understood from the approximate separation of the set of two-exciton states, into classes of inter-ring and intra-ring two-exciton states. Our calculations show that the experimental linear absorption spectrum may be used to estimate the cylinder circumference, while the pump-probe spectrum yields information on the length of the cylinder or on the delocalization length of the excitons in its longitudinal direction. We apply this method to cylindrical aggregates of cyanine dyes.