We report the selective area growth of InP layers in submicron trenches on Si (001) substrates by using a thin Ge buffer layer. The antiphase domain boundaries in InP layers are suppressed by engineering the local Ge surface profile. The mechanism of atomic step formation and the corresponding method for step density control are presented. We discuss the impact of the surface profile of the Ge buffer layer on the formation of antiphase domain boundaries as well as on InP nucleation. A minimum step density of 0.25 nm(-1) is required to avoid antiphase domain boundaries while a higher step density substantially reduces the stacking faults and twins in the InP nucleation layer. By employing the threading dislocation necking effect and the properly controlled Ge surface profile, high-quality InP layers have been obtained in submicron trenches. (C) 2011 The Electrochemical Society. [DOI: 10.1149/1.3571248] All rights reserved.