We consider a robust state estimation problem for time-varying uncertain discrete-time, homogeneous,first-order,finite-state finite-alphabet hidden Markov models (HMMs). A class of time-varying uncertain HMMs is considered in which the uncertainty is sequentially described by a conditional relative entropy constraint on perturbed conditional probability measures given a realized observation sequence. For this class of uncertain HMMs, the robust state estimation problem is formulated as a constrained optimization problem. Using a Lagrange multiplier technique and a variational formula for conditional relative entropy, the above problem is converted into an unconstrained optimization problem and a problem related to partial information risk-sensitive filtering. A measure transformation technique and an information state method are employed to solve this equivalent problem related to risk-sensitive filtering. A characterization of the solution to the robust state estimation problem is also presented.