Gene silencing is essential for regulating cell fate in eukaryotes. Altered chromatin architectures contribute to maintaining the silenced state in a variety of species. The silent information regulator (Sir) proteins regulate mating type in Saccharomyces cerevisiae. One of these proteins, Sir3, interacts directly with the nucleosome to help generate silenced domains. We determined the crystal structure of a complex of the yeast Sir3 BAH (bromo-associated homology) domain and the nucleosome core particle at 3.0 angstrom resolution. We see multiple molecular interactions between the protein surfaces of the nucleosome and the BAH domain that explain numerous genetic mutations. These interactions are accompanied by structural rearrangements in both the nucleosome and the BAH domain. The structure explains how covalent modifications on H4K16 and H3K79 regulate formation of a silencing complex that contains the nucleosome as a central component.