Ultimately, numerical simulation of viscoelastic flows will prove most useful if the calculations can predict the details of steady-state processing conditions as well as the linear stability and non-linear dynamics of these states. We use finite element spatial discretization coupled with a semi-implicit theta-method for time integration to explore the linear and non-linear dynamics of two, two-dimensional viscoelastic flows: plane Couette flow and pressure-driven flow past a Linear, periodic array of cylinders in a channel. For the upper convected Maxwell (UCM) fluid, the linear stability analysis for the plane Couette flow can be performed in closed form and the two most dangerous, although always stable, eigenvalues and eigenfunctions are known in closed form. The eigenfunctions are non-orthogonal in the usual inner product and hence, the linear dynamics are expected to exhibit non-normal (non-exponential) behavior at intermediate times. This is demonstrated by numerical integration and by the definition of a suitable growth function based on the eigenvalues and the eigenvectors. Transient growth of the disturbances at intermediate times is predicted by the analysis for the UCM fluid and is demonstrated in linear dynamical simulations for the Oldroyd-B model. Simulations for the fully non-linear equations show the amplification of this transient growth that is caused by non-linear coupling between the non-orthogonal eigenvectors. The finite element analysis of linear stability to two-dimensional disturbances is extended to the two-dimensional flow past a linear, periodic array of cylinders in a channel, where the steady-state motion itself is known only from numerical calculations. For a single cylinder or widely separated cylinders, the flow is stable for the range of Deborah number (De) accessible in the calculations. Moreover, the dependence of the most dangerous eigenvalue on De = lambda V/R resembles its behavior in simple shear flow, as does the spatial structure of the associated eigenfunction. However, for closely spaced cylinders, an instability is predicted with the critical Deborah number De(c) scaling linearly with the dimensionless separation distance L between the cylinders, that is, the critical Deborah number De(Lc) = lambda V/L is shown to be an O(1) constant. The unstable eigenfunction appears as a family of two-dimensional vortices close to the channel wall which travel downstream. This instability is possibly caused by the interaction between a shear mode which approaches neutral stability for De much greater than 1 and the periodic modulation caused by the presence of the cylinders. Nonlinear time-dependent simulations show that this secondary flow eventually evolves into a stable limit cycle, indicative of a supercritical Hopf bifurcation from the steady base state.