A systematic investigation of crack formation has been performed in tensile InxGa1-xAs/InP layers with indium composition ranging from x = 0.2 up to x = 0.35 and thicknesses ranging from 8 nm to 2 mum. It has been found that cracks form after growth and on a characteristic timescale of several days. The formation of cracks has been found to occur in a well defined thickness interval correlated to the residual strain during growth. Crack formation is favored along the [110] in-plane direction in samples with low indium composition. The results can be rationalized within a model which explicitly takes into account the fact that cracks form after growth.