The magnetic and crystal structures of the oxygen-deficient layered perovskite YBaCuFeO5+DELTA have been determined with powder neutron diffraction methods. It crystallizes in the noncentrosymmetric tetragonal space group P4mm with a = 3.8707(2) angstrom and c = 7.6711(4) angstrom (at room temperature). Structural motifs of corner-sharing severely elongated CuO5 (Cu-O(eq) = 1.936(2) angstrom, Cu-O(ax) = 2.35(9) angstrom) and slightly squashed FeO5 (Fe-O(eq) = 2.03(3) angstrom, Fe-O(ax) = 1.82(9) angstrom) square pyramids result in a novel lamellar structure with ordered oxygen defects. Local magnetic moments directed perpendicular to the unique axis (mu perpendicular to c, mu = 2.51(15) mu(B) at room temperature) appear at the Fe3+ sites at 442.4(5) K. The magnetic unit cell results by doubling the chemical one along all three crystallographic directions (magnetic propagation vector, k = (1/2,1/2,1/2)). Despite the long separation between successive FeO2 layers (approximately 7.7 angstrom), the magnetic behavior can be successfully described by a three-dimensional Heisenberg spin Hamiltonian with a critical exponent beta = 0.366(8). No evidence of ordering of the Cu sublattice is found on cooling to room temperature.