We study the effect of the osmolyte, Trimethylamine N-Oxide (TMAO), which accumulates in cells in response to osmotic stress, on the stability of RNA hairpins. All atom molecular dynamics (MD) simulations of a nucleotide and the 22-nucleotide RNA hairpin P5GA in an aqueous TMAO solution show that TMAO preferentially interacts with the base through the formation of a single hydrogen bond. To circumvent the difficulties of adequately sampling the conformational space of polynucleotides, we used coarse-grained models (including one that is inspired by the results, of all-atom MD simulations of a single nucleotide) to probe the effects of osmoyltes on the stability of P5GA. If, as revealed by our MD simulations, the cosolute specifically interacts with only one base at a time, then we find practically no change in hairpin stability as measured by Delta T-m = T-m(Phi) - T-m, where T-m(Phi) and T-m are the melting temperatures at volume fraction Phi of the osmolyte and Phi = 0, respectively. This finding is in qualitative agreement with recent experiments. If the interactions between the RNA and osmolytes are repulsive, which is appropriate for mimicking the effects of crowding, Delta T-m can vary from 5 to 15 K depending on the size of the osmolyte and the nature of RNA-osmolyte interactions. Cosolutes that interact favorably with multiple bases simultaneously can stabilize the hairpin more than a crowding agent of the same size. The implications of our predictions for experiments are briefly outlined.