When a capillary tube is brought into contact with a wetting liquid, surface tension forces overcome gravity and the liquid spontaneously rises into the tube until an equilibrium height is reached. The early viscous dynamics of the rise typically follow the well-known Lucas-Washburn law, which is independent of gravity and neglects the displaced fluid. Here we explore the early viscous dynamics when the properties, of displaced fluid are significant. Using a combination of experiments and theory, we show how the characteristic behavior of the Lucas-Washburn law is modified when the viscosity of the displaced fluid is comparable to or exceeds the wetting fluid. Additionally, we find that the effects of gravity reshape the dynamics of the capillary rise not only in the late viscous regime but also in the early viscous regime.