On tight bounds for function approximation error in risk-sensitive reinforcement learning

Abstract

In this letter we provide several informative tight error bounds when using value function approximators for the risk-sensitive cost setting for a given policy represented using exponential utility. The novelty of our approach is that we make use of the irreducibility of the underlying Markov chain (resulting in better bounds using Perron–Frobenius eigenvectors) to derive new bounds whereas the earlier work used primarily the spectral variation bound which holds for any matrix, hence did not make use of the irreducibility. All our bounds have a perturbation term for large state spaces. We also present examples where we show that the new bounds perform 90-100% better than the earlier proposed spectral variation bound.