Prashanth, L.A. ; Bhatnagar, Shalabh ; Bhavsar, Nirav ; Fu, Michael ; Marcus, Steven I. (2020) Random Directions Stochastic Approximation With Deterministic Perturbations IEEE Transactions on Automatic Control, 65 (6). pp. 2450-2465. ISSN 0018-9286
Full text not available from this repository.
Official URL: http://doi.org/10.1109/TAC.2019.2930821
Related URL: http://dx.doi.org/10.1109/TAC.2019.2930821
Abstract
We introduce deterministic perturbation (DP) schemes for the recently proposed random directions stochastic approximation, and propose new first-order and second-order algorithms. In the latter case, these are the first second-order algorithms to incorporate DPs. We show that the gradient and/or Hessian estimates in the resulting algorithms with DPs are asymptotically unbiased, so that the algorithms are provably convergent. Furthermore, we derive convergence rates to establish the superiority of the first-order and second-order algorithms, for the special case of a convex and quadratic optimization problem, respectively. Numerical experiments are used to validate the theoretical results.
| Item Type: | Article |
|---|---|
| Source: | Copyright of this article belongs to Institute of Electrical and Electronics Engineers. |
| Keywords: | Random Directions Stochastic Approximation (RDSA); Simultaneous Perturbation Stochastic Approximation (SPSA); Stochastic Approximation; Stochastic Optimization. |
| ID Code: | 116437 |
| Deposited On: | 12 Apr 2021 05:52 |
| Last Modified: | 12 Apr 2021 05:52 |
Repository Staff Only: item control page
