In directional solidification of binary alloys, a dendritic mush is formed between the bulk melt above and the eutectic solid below. We investigate the convective instabilities in such a three-layer system during solidification process, which differs from previous two-layer model by the solid layer. Due to the existence of the solid layer, the mush/solid interface is allowed to deform so that the mushy layer becomes less stable while the stability of the melt is virtually unchanged. The mushy layer also becomes less stable when the latent heat of fusion generated in the mush is larger, while the stability of the melt above is not affected because the heat is transferred through the solid layer towards the cooling boundary below. Present results may help reflect the reality that the heat transferring to the cooling boundary decreases with time so that the stability of the mush will be enhanced during solidification process, and explain the discrepancy between former theoretical predictions and experimental measurements. (C) 2003 Elsevier Science B.V. All rights reserved.