The elastic viscous split stress formulation (EVSS) and the discrete EVSS formulation (DEVSS) are effective in stabilizing numerical simulations of viscoelastic flows and have been widely used. Following the concept of Galerkin least-square perturbations proposed by Hughes et al. [Comput. Meth Appl. Mech. Eng. 73 (1989) 173-189] and Franca et al. [SIAM J. Numer. Anal. 28(6) (1991) 1680-1697; Comput. Meth. Appl. Mech. Eng. 99 (1992) 209-233; Ibid. 104 (1993) 31-48] we are able to give the DEVSS formulation a new explanation as a perturbation to the Galerkin method based on the strain-rate residual, and furthermore, introduce another stabilized formulation, here named as MIX1, based on the incompressibility residual of the finite element discretizations. The three formulations (EVSS, DEVSS, MIX1), combined with a h-p type finite element algorithm that employs the SUPG technique to solve the viscoelastic constitutive equations are then tested on three benchmark problems: the flow of the upper-convected Maxwell fluid between eccentric cylinders, the flow of the Maxwell fluid around a sphere in a tube and the flow of the Maxwell and Oldroyd-B fluids around a cylinder in a channel. The results are checked with previous published works; good agreement is observed. Our numerical experiments convincingly demonstrate that the MIX1 is an accurate algorithm and convergent in terms of the p-extension, it has the same level of stability and robustness as the DEVSS method and is superior to the EVSS method in some respects. More important is that with MIX1 method one needs not solve for the strain-rate tensor as in EVSS and DEVSS methods, therefore, the CPU time consumption in the MIX1 method especially when using a coupled iteration scheme can be radically reduced. The success of the MIX1 method presents a challenge to the widely accepted concept of making the momentum equation explicitly elliptic.