This paper extends the results of Masreliez [8] on the design of non-Gaussian estimators for a more general class of the parameter estimation problem when the system state and the observation noise may be dependent and non-Gaussian simultaneously It is shown that the proposed non-Gaussian algorithms can approximate with high precision the minimum mean square estimator. Application of the approach to the design of different optimal (and stable) estimation algorithms is illustrated. The efficiency of the proposed algorithms is tested in some simulation experiments.