The existence and stability of elastic overstability in the presence of non-negligible inertia for an Oldroyd-B fluid are examined for the Rayleigh-Bernard thermal convection problem. The study is based on the four-dimensional non-linear dynamical system presented by Khayat (J. Non-Newtonian Fluid Mech., 53 (1994) 227) which constitutes a generalization of the classical Lorenz system for a Newtonian fluid. It is shown that elastic overstability can only set in once the Deborah number exceeds a critical value which depends on the Prandtl number and fluid retardation. Fluid elasticity is found to precipitate the onset of overstability while retardation tends to delay it. The conditions of existence of the corresponding Hopf bifurcation are examined as functions of fluid elasticity, retardation and thermal conductivity. The stability of the periodic orbit (in phase space) is investigated using center manifold theory. It is found that the orbit is asymptotically stable to perturbations about the conductive state, with the initial period of oscillation decreasing with Deborah number, reaching a minimum, and increasing asymptotically toward a constant value.