Superhydrophobic surfaces often incorporate roughness on both micron and nanometer length scales, although a satisfactory understanding of the role of this hierarchical roughness in causing sup erhydrophobicity remains elusive. We present a two-dimensional thermodynamic model to describe wetting on hierarchically grooved surfaces by droplets for which the influence of gravity is negligible. By creating wetting phase diagrams for droplets on surfaces with both single-scale and hierarchical roughness, we find that hierarchical roughness leads to greatly expanded super-hydrophobic domains in phase space over those for a single scale of roughness. Our results indicate that an important role of the nanoscale roughness is to increase the effective Young's angle of the microscale features, leading to smaller required aspect ratios (height to width) for the surface structures. We then show how this idea may be used to design a hierarchically rough surface with optimally high contact angles.