Adaptive control of Markov chains. I. Finite parameter set

Borkar, V. ; Varaiya, P. (1979) Adaptive control of Markov chains. I. Finite parameter set Automatic Control, IEEE Transactions on, 24 (6). pp. 953-957. ISSN 0018-9286

Full text not available from this repository.

Official URL:

Related URL:


Consider a controlled Markov chain whose transition probabilities depend upon an unknown parameter a taking values in finite setA. To each a is associated a prespecified stationary control law φ(α). The adaptive control law selects at each timetthe control action indicated byphi(alpha_{t})where atis the maximum likelihood estimate of α. It is shown that atconverges to a parameter α*such that the "closed-loop" transition probabilities corresponding to α*and φ(α^{ast}) are the same as those corresponding to α0 and φ(α)where α0 is the true parameter. The situation when a0does not belong to the model setAis briefly discussed.

Item Type:Article
Source:Copyright of this article belongs to IEEE.
ID Code:81409
Deposited On:06 Feb 2012 04:26
Last Modified:06 Feb 2012 04:26

Repository Staff Only: item control page