Dominating finite-range interactions in many-body systems can lead to intriguing self-ordered phases of matter. For quantum magnets, Ising models with power-law interactions are among the most elementary systems that support such phases. These models can be implemented by laser coupling ensembles of ultracold atoms to Rydberg states. Here, we report on the experimental preparation of crystalline ground states of such spin systems. We observe a magnetization staircase as a function of the system size and show directly the emergence of crystalline states with vanishing susceptibility. Our results demonstrate the precise control of Rydberg many-body systems and may enable future studies of phase transitions and quantum correlations in interacting quantum magnets.