The analytical and numerical solutions of the non-linear exact Kushner filter are not possible, since the mean and variance evolutions are infinite dimensional and require the knowledge of the higher-order moment evolutions. The approximate filters seem to preserve some of the qualitative characteristics of the exact filter. In this paper, evolutions of conditional mean and conditional covariance of the third-order approximate filter for estimating the states of a non-linear dynamical system, especially accounting state-dependent and state-independent noise perturbations, are derived. In this analysis, we make a comparison of this filter with second-order Gaussian filter discussed in standard textbooks on non-linear filtering. This paper discusses Duffing filter, by taking up two different non-linear observation equations to demonstrate the effectiveness of the higher-order filters, i.e. third order, and second-order filters. Most notably, this paper is about examining the ability of the higher-order filters for estimating the states of the stochastically perturbed non-linear dynamical systems. In addition, the analytical findings are supported with numerical work generated by a simple, but effective, finite difference scheme.