Exciton-polaritons are hybrid light-matter quasiparticles formed by strongly interacting photons and excitons (electron-hole pairs) in semiconductor microcavities(1-3). They have emerged as a robust solid-state platform for next-generation optoelectronic applications as well as for fundamental studies of quantum many-body physics. Importantly, exciton-polaritons are a profoundly open (that is, non-Hermitian(4,5)) quantum system, which requires constant pumping of energy and continuously decays, releasing coherent radiation(6). Thus, the exciton-polaritons always exist in a balanced potential landscape of gain and loss. However, the inherent non-Hermitian nature of this potential has so far been largely ignored in exciton-polariton physics. Here we demonstrate that non-Hermiticity dramatically modifies the structure of modes and spectral degeneracies in exciton-polariton systems, and, therefore, will affect their quantum transport, localization and dynamical properties(7-9). Using a spatially structured optical pump(10-12), we create a chaotic exciton-polariton billiard-a two-dimensional area enclosed by a curved potential barrier. Eigenmodes of this billiard exhibit multiple non-Hermitian spectral degeneracies, known as exceptional points(13,14). Such points can cause remarkable wave phenomena, such as unidirectional transport(15), anomalous lasing/absorption(16,17) and chiral modes(18). By varying parameters of the billiard, we observe crossing and anti-crossing of energy levels and reveal the non-trivial topological modal structure exclusive to non-Hermitian systems(9,13-22). We also observe mode switching and a topological Berry phase for a parameter loop encircling the exceptional point(23,24). Our findings pave the way to studies of non-Hermitian quantum dynamics of exciton-polaritons, which may uncover novel operating principles for polariton-based devices.