Immiscible polymer blends are known to display an unusual elastic recovery after stress release. Recoil after steady-state shearing is well understood and obeys specific scaling relations. Releasing the stress before the steady-state morphology has been reached results in a more complex elastic recovery, including very large final values. This behaviour is investigated systematically. Model blends are used, consisting of nearly inelastic components; hence the measured recoil can be attributed totally to contributions from the interface. The instantaneous structure at the onset of the recoil can vary greatly in transient experiments, ranging from slightly deformed droplets to highly elongated filaments. The effects of this initial structure on the ultimate recoil and time scale of the recovery are studied. The morphological changes during recovery are considered as well. It is demonstrated that they can be computed from the normal stresses during stress relaxation with comparable initial morphologies. This indicates that the same morphological changes occur during stress relaxation and constrained recoil. A scaling relation for the recoil curves has been derived from the Doi-Ohta theory, which is confirmed by the experiments.