The conformation of cylindrical brush polymers with a polymethacrylate main and polystyrene side chains (6 <= P-n(sc) <= 33, with P-n(sc) the number-average degree of polymerization of the side chains) were studied by combined light and small-angle neutron scattering experiments. The results reveal that the main chain stiffness expressed in terms of the Kuhn statistical segment length, l(k), increases with side chain length but does not follow scaling predictions which most probably is due to the limited length of the side chains investigated experimentally. In this respect the present work addresses the transition regime from flexible coils to stiff cylindrical brushes as a function of side chain length. In detail, the increase of lk is stronger in toluene, a very good solvent for the side chains, than in the poor solvent cyclohexane and does not level off for the longest side chains investigated (P-n(sc) = 33). In contrast to earlier work, the cylinder length per main chain monomer is found to be independent of side chain length but to depend slightly on the solvent quality, i.e., l(m) = 0.241 nm in toluene and l(m) = 0.207 nm in cyclohexane. The value determined in toluene is close to the maximum value of lm = 0.25 nm expected for a fully stretched vinylic main chain, whereas the smaller value for lm in cyclohexane suggests a local coiling of the main chain, most probably caused by less repulsive interactions between the side chains. The discrepancy to some earlier scattering experiments could be resolved, but the origin of frequently reported much smaller cylinder lengths derived by atomic force microscopy remains unclear.