Linear and weakly nonlinear properties of thermohaline convection in rotating fluids are investigated. Linear stability analysis is studied by plotting graphs for different values of physical parameters relevant to the Earth's outer core and oceans. We have derived a nonlinear two-dimensional Landau-Ginzburg equation with real coefficients near the onset of stationary convection at the supercritical pitchfork bifurcation and shown the occurrence of Eckhaus and zigzag instabilities. We have studied heat transfer by using Nusselt number which is obtained from Landau-Ginzburg equation at the onset of stationary convection for the steady case. A coupled two-dimensional Landau-Ginzburg type equations with complex coefficients near the onset of oscillatory convection are derived and the stability regions of travelling and standing waves discussed. (C) 2007 Elsevier Ltd. All rights reserved.