A father's lifetime experiences can be transmitted to his offspring to affect health and development. However, the mechanisms underlying paternal epigenetic transmission are unclear. Unlike in somatic cells, there are few nucleosomes in sperm, and their function in epigenetic inheritance is unknown. We generated transgenic mice in which overexpression of the histone H3 lysine 4 (H3K4) demethylase KDM1A (also known as LSD1) during spermatogenesis reduced H3K4 dimethylation in sperm. KDM1A overexpression in one generation severely impaired development and survivability of offspring. These defects persisted transgenerationally in the absence of KDM1A germline expression and were associated with altered RNA profiles in sperm and offspring. We show that epigenetic inheritance of aberrant development can be initiated by histone demethylase activity in developing sperm, without changes to DNA methylation at CpG-rich regions.