Dynamic programming for ergodic control with partial observations

Abstract

A dynamic programming principle is derived for a discrete time Markov control process taking values in a finite dimensional space, with ergodic cost and partial observations. This uses the embedding of the process into another for which an accessible atom exists and hence a coupling argument can be used. In turn, this is used for deriving a martingale dynamic programming principle for ergodic control of partially observed diffusion processes, by ‘lifting’ appropriate estimates from a discrete time problem associated with it to the continuous time problem.