Single crystals of the stoichiometric iron calcium Oxysulfide CaOFeS have been grown by a solid-state reaction. Structural analysis of CaOFeS at room temperature by combining single-crystal X-ray diffraction data and transmission electron microscopy leads to a stoichiometric hexagonal noncentrosymmetric P6(3)mc layered structure isostructural to CaOZnS. It is built from alternating layers made of FeOS3 tetrahedra sharing sulfur apexes and stacked with Ca2+ planes. All Fe-O bonds are parallel to the stacking axis; this breaks the centrosymmetry, leading to a polar structure. The dielectric measurements reveal the existence of a magnetodielectric effect near 33 K in good agreement with the Neel temperature, as evidenced near 35 K by specific heat measurements reported by a different group.