Recent trends in Markov decision processes

Abstract

A short tutorial introduction is given to Markov decision processes (MDP), including the latest activity in the field. Classical theory, most notably dynamic programming and its consequences, is briefly reviewed along with MDP computational techniques for MDP. A brief overview is given of issues that remain open in the classical framework. Some 'nonclassical' problems that have attracted much attention recently are addressed, including problems involving nonclassical costs and multiobjective problems. Problems involving partial information about the state of the process or about model uncertainty are considered. Some application areas and the closely related areas of continuous time stochastic control and stochastic games are addressed. (AIAA).