Crystal and magnetic structures of BiMnO3+delta (delta = 0.03, 0.08, and 0.14) have been determined by the Rietveld method from neutron diffraction data at 8-10 and 290 K. BiMnO3.03 (= Bi0.99Mn0.99O3) crystallizes in a monoclinic system (the refinement was performed in space group C2/c; Z= 8; a = 9.5313(3) angstrom, b = 5.57791(17) angstrom, c = 9.7375(4) angstrom, beta = 108.951(2)degrees at 290 K). BiMnO3.08 (= Bi0.974Mn0.974O3) crystallizes in space group P2(1)/c (Z= 8; a = 9.5565(4) angstrom, b = 5.51823(16) angstrom, c = 9.7051(4) angstrom, beta = 109.442(3)degrees at 290 K). It was found that Mn vacancies are localized mainly in one Mn site (among three sites) in Bi0.974Mn0.974O3. Vacancy-ordering and charge-ordering scenarios are suggested as possible reasons for the crystal symmetry change compared with Bi0.99Mn0.99O3. BiMnO3.03 and BiMnO3.08 are ferromagnetic below T-C = 82 and 68 K, respectively, with magnetic moments along the monoclinic b axes. Refined magnetic moments at 10 K are 2.88(2)mu(B) in BiMnO3.03 and 2.33(2)mu(B) in BiMnO3.08. BiMnO3.14 (= Bi0.955Mn0.955O3) crystallizes in an orthorhombic system (space group Pnma; Z = 4; a = 5.5136(4) angstrom, b = 7.8069(8) angstrom, and c = 5.5454(5) angstrom at 290 K), and its structure is similar to that of LaMnO3.11-LaMnO3.15. No magnetic reflections were found in BiMnO3.14 down to 8 K, in agreement with its spin-glass magnetic state. Magnetic and chemical properties of BiMnO3+delta (0.02 <= delta <= 0.14) have also been investigated and compared with those of LaMnO3+delta. Systematic changes of magnetic parameters in BiMnO3+delta were found to depend on delta.