Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences

Abstract

Simultaneous perturbation stochastic approximation (SPSA) algorithms have been found to be very effective for high-dimensional simulation optimization problems. The main idea is to estimate the gradient using simulation output performance measures at only two settings of the N-dimensional parameter vector being optimized rather than at the N + 1 or 2N settings required by the usual one-sided or symmetric difference estimates, respectively. The two settings of the parameter vector are obtained by simultaneously changing the parameter vector in each component direction using random perturbations. In this article, in order to enhance the convergence of these algorithms, we consider deterministic sequences of perturbations for two-timescale SPSA algorithms. Two constructions for the perturbation sequences are considered: complete lexicographical cycles and much shorter sequences based on normalized Hadamard matrices. Recently, one-simulation versions of SPSA have been proposed, and we also investigate these algorithms using deterministic sequences. Rigorous convergence analyses for all proposed algorithms are presented in detail. Extensive numerical experiments on a network of M/G/1 queues with feedback indicate that the deterministic sequence SPSA algorithms perform significantly better than the corresponding randomized algorithms.