SIAM Journal on Control and Optimization, Vol.40, No.2, 348-369, 2001
Limiting discounted-cost control of partially observable stochastic systems
This paper presents two main results on partially observable (PO) stochastic systems. In the rst one, we consider a general PO system x(t+1) = F(x(t), a(t), xi (t)), y(t) = G(x(t), eta (t)) (t = 0,1,...) (*) on Borel spaces, with possibly unbounded cost-per-stage functions, and we give conditions for the existence of alpha -discount optimal control policies (0<<1). In the second result we specialize () to additive-noise systems x(t + 1) = F-n(x(t), a(t)) + (t), y(t) = G(n)(x(t)) + eta (t) (t = 0,1,...) in Euclidean spaces with F-n (x, a) and G(n) (x) converging pointwise to functions F-infinity (x, a) and G(infinity) (x), respectively, and we give conditions for the limiting PO model x(t + 1) = F-infinity(x(t), a(t)) + xi (t), y(t) = G(infinity)(x(t)) + eta (t) to have an alpha -discount optimal policy.
Keywords:partially observable control systems;partially observable Markov control processes;hidden Markov models;discounted cost criterion