In selective-area metalorganic vapor-phase epitaxy, polycrystals are generated on masks of large area. The generation of such crystals should be avoided because they act as uncontrollable sink of film precursors. Although generation of polycrystals is sometimes associated with surface migration of a precursor on masks, our observation suggested that polycrystals are generated when the gas-phase concentration of a film precursor just above masks exceeds a critical value. Reducing either total pressure or partial pressure of the source gas suppressed generation of polycrystals, which is consistent with the gas-phase mechanism. The threshold of gas-phase precursor concentration for generation of polycrystals was investigated as a function of growth temperature and V/III ratio for both GaAs and InP. The threshold increased with temperature, that is to say, fewer polycrystals were observed at higher temperature. The threshold for InP was significantly larger than that for GaAs. This information is useful for designing mask patterns which do not generate polycrystals, and it helps to provide a detailed understanding of nucleation phenomena. (C) 2003 Elsevier B.V. All rights reserved.