Ergodic and adaptive control of nearest-neighbor motions

Abstract

The self-tuning approach to adaptive control is applied to a class of Markov chains called nearest-neighbor motions. These have a countable state space and move from any state to at most finitely many neighboring states. For compact parameter and control spaces, the almost-sure optimality of the self-tuner for an ergodic cost criterion is established under two sets of assumptions.