We examine the effects of matrix phase viscoelasticity on the rheological modeling of polymer blends with a droplet morphology. Two contravariant, second-rank tensor variables are adopted along with the translational momentum density of the fluid to account for viscoelasticity of the matrix phase and the ellipsoidal droplet shapes. The first microstructural variable is a conformation tensor describing the average extension and orientation of the molecules in the matrix phase. The other microstructural variable is a configuration tensor to account for the average shape and orientation of constant-volume droplets. A Hamiltonian framework of non-equilibrium thermodynamics is then adopted to derive a set of continuum equations for the system variables. This set of equations accounts for local conformational changes of the matrix molecules due to droplet deformation and vice versa. The model is intended for dilute blends of both oblate and prolate droplets, and droplet breakup and coalescence are not taken into account. Only the matrix phase is considered as viscoelastic; i.e., the droplets are assumed to be Newtonian. The model equations are solved for various types of homogeneous deformations, and microstructure/rheology relationships are discussed for transient and steady-state conditions. A comparison with other constrained-volume rheological models and experimental data is made as well.