In this article, an investigation is conducted to analyze the marginal stability with and without magnetic field in a mushy layer. During alloy solidification, such mushy layer, which is adjacent to the solidification front and composed of solid dendrites and liquid, is known to produce vertical chimneys. Here, we carry out numerical investigation for particular range of parameter values, which cover those of available experimental studies, to determine the convective flow at the onset of motion. The governing coupled non-linear partial differential equations are non-dimensionalised and solved to get the steady basic-state solution. The thickness of the mushy layer is determined as a part of the solution. Using multiple shooting technique, we determine the steady-state solutions in a range of critical Rayleigh number. We analyse the effect of several parameters, Chandrasekhar number Q, and Robert's number tau on the problem. It was found that an increase in Q has a stabilizing effect on solidification and the critical Rayleigh number increases on increasing Q. It was also found that for moderate or small values of Robert's number tau the critical Rayleigh number is mostly insensitive.