The critical dimensions for the introduction of interfacial misfit dislocations into epitaxially grown In0.6Ga0.4As islands on GaAs(001) substrates are investigated by high-resolution transmission electron microscopy. By the combination of the continuum theory of elasticity with numerical finite element simulations, equations are derived in order to calculate a phase diagram separating the coherent from the incoherent phase in dependence on the island size. It is demonstrated that, if the elastic relaxation of the laterally limited epilayers is taken into account in a quantitative manner, the derived expressions for the coherent-to-incoherent transition are in excellent agreement with the experimental observations. When geometrical and material parameters are modified correspondingly the model calculations may also be applied to the analysis of other heteroepitaxial systems for which dislocation formation is not kinetically limited.