Browse by Fellow

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh (2022) Analyzing Approximate Value Iteration Algorithms Mathematics of Operations Research, 47 (3). pp. 2138-2159. ISSN 0364-765X

Diddigi, Raghuram Bharadwaj ; Kamanchi, Chandramouli ; Bhatnagar, Shalabh (2022) A Generalized Minimax Q-Learning Algorithm for Two-Player Zero-Sum Stochastic Games IEEE Transactions on Automatic Control, 67 (9). pp. 4816-4823. ISSN 0018-9286

Kamanchi, Chandramouli ; Diddigi, Raghuram Bharadwaj ; Bhatnagar, Shalabh (2022) Generalized Second-Order Value Iteration in Markov Decision Processes IEEE Transactions on Automatic Control, 67 (8). pp. 4241-4247. ISSN 0018-9286

Singla, Abhik ; Padakandla, Sindhu ; Bhatnagar, Shalabh (2021) Memory-Based Deep Reinforcement Learning for Obstacle Avoidance in UAV With Limited Environment Knowledge IEEE Transactions on Intelligent Transportation Systems, 22 (1). pp. 107-118. ISSN 1524-9050

J., Prabuchandran K. ; Penubothula, Santosh ; Kamanchi, Chandramouli ; Bhatnagar, Shalabh (2021) Novel First Order Bayesian Optimization with an Application to Reinforcement Learning Applied Intelligence, 51 (3). pp. 1565-1579. ISSN 0924-669X

Karmakar, Prasenjit ; Bhatnagar, Shalabh (2021) On tight bounds for function approximation error in risk-sensitive reinforcement learning Systems & Control Letters, 150 . p. 104899. ISSN 0167-6911

Karmakar, Prasenjit ; Bhatnagar, Shalabh (2021) Stochastic approximation with iterate-dependent Markov noise under verifiable conditions in compact state space with the stability of iterates not ensured IEEE Transactions on Automatic Control . p. 1. ISSN 0018-9286

Yaji, Vinayaka G. ; Bhatnagar, Shalabh (2020) Analysis of Stochastic Approximation Schemes With Set-Valued Maps in the Absence of a Stability Guarantee and Their Stabilization IEEE Transactions on Automatic Control, 65 (3). pp. 1100-1115. ISSN 0018-9286

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh ; Quevedo, Daniel E. (2020) Asynchronous stochastic approximations with asymptotically biased errors and deep multi-agent learning IEEE Transactions on Automatic Control . p. 1. ISSN 0018-9286

John, Indu ; Bhatnagar, Shalabh (2020) Deep Reinforcement Learning with Successive Over-Relaxation and its Application in Autoscaling Cloud Resources In: International Joint Conference on Neural Networks (IJCNN), 19-24 July 2020, Glasgow, UK.

John, Indu ; Kamanchi, Chandramouli ; Bhatnagar, Shalabh (2020) Generalized Speedy Q-Learning IEEE Control Systems Letters, 4 (3). pp. 524-529. ISSN 2475-1456

Dharmavaram, Akshay ; Riemer, Matthew ; Bhatnagar, Shalabh (2020) Hierarchical Average Reward Policy Gradient Algorithms (Student Abstract) In: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), Feb 7-12, 2020, New York, NY, USA.

Tirumala, Sashank ; Gubbi, Sagar ; Paigwar, Kartik ; Sagi, Aditya ; Joglekar, Ashish ; Bhatnagar, Shalabh ; Ghosal, Ashitava ; Amrutur, Bharadwaj ; Kolathaya, Shishir (2020) Learning Stable Manoeuvres in Quadruped Robots from Expert Demonstrations In: 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 31 Aug.-4 Sept. 2020, Naples, Italy.

Padakandla, Sindhu ; Rao, Shilpa ; Bhatnagar, Shalabh (2020) Learning-Based Resource Allocation in Industrial IoT Systems In: IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, 31 Aug.-3 Sept. 2020, London, UK.

Prashanth, L.A. ; Bhatnagar, Shalabh ; Bhavsar, Nirav ; Fu, Michael ; Marcus, Steven I. (2020) Random Directions Stochastic Approximation With Deterministic Perturbations IEEE Transactions on Automatic Control, 65 (6). pp. 2450-2465. ISSN 0018-9286

Padakandla, Sindhu ; K. J., Prabuchandran ; Bhatnagar, Shalabh (2020) Reinforcement learning algorithm for non-stationary environments Applied Intelligence, 50 (11). pp. 3590-3606. ISSN 0924-669X

Nayak, Shravan ; Ekbote, Chanakya Ajit ; Pratap Singh Chauhan, Annanya ; Diddigi, Raghuram Bharadwaj ; Ray, Prishita ; Sikdar, Abhinava ; Reddy Danda, Sai Koti ; Bhatnagar, Shalabh (2020) Stochastic Game Frameworks for Efficient Energy Management in Microgrid Networks In: IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe), The Hague, Netherlands, 26-28 Oct. 2020.

Yaji, Vinayaka G. ; Bhatnagar, Shalabh (2020) Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise Mathematics of Operations Research, 45 (4). pp. 1405-1444. ISSN 0364-765X

Kamanchi, Chandramouli ; Diddigi, Raghuram Bharadwaj ; Bhatnagar, Shalabh (2020) Successive Over-Relaxation ${Q}$ -Learning IEEE Control Systems Letters, 4 (1). pp. 55-60. ISSN 2475-1456

Diddigi, Raghuram Bharadwaj ; Reddy, D. Sai Koti ; K.J., Prabuchandran ; Bhatnagar, Shalabh (2019) Actor-Critic Algorithms for Constrained Multi-agent Reinforcement Learning In: 18th International Conference on Autonomous Agents and Multiagent Systems, May 2019, Montreal, QC, Canada.

Joseph, Ajin George ; Bhatnagar, Shalabh (2019) An Adaptive Sampling Algorithm for Policy Evaluation In: Fifth Indian Control Conference (ICC), 9-11 Jan. 2019, New Delhi, India.

Joseph, Ajin George ; Bhatnagar, Shalabh (2019) An Adaptive and Incremental Approach to Quantile Estimation In: IEEE 58th Conference on Decision and Control (CDC), 11-13 Dec. 2019, Nice, France.

Dholakiya, Dhaivat ; Bhattacharya, Shounak ; Gunalan, Ajay ; Singla, Abhik ; Bhatnagar, Shalabh ; Amrutur, Bharadwaj ; Ghosal, Ashitava ; Kolathaya, Shishir (2019) Design, Development and Experimental Realization of A Quadrupedal Research Platform: Stoch In: 5th International Conference on Control, Automation and Robotics (ICCAR), 19-22 April 2019, Beijing, China.

John, Indu ; Bhatnagar, Shalabh (2019) Efficient Budget Allocation and Task Assignment in Crowdsourcing In: CoDS-COMAD '19: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Jan 3-5, 2019, Kolkata, India.

Joseph, Ajin George ; Bhatnagar, Shalabh (2019) An Incremental Algorithm for Estimating Extreme Quantiles In: Sixth Indian Control Conference (ICC), 18-20 Dec. 2019, Hyderabad, India.

Bhattacharya, Shounak ; Singla, Abhik ; Abhimanyu, . ; Dholakiya, Dhaivat ; Bhatnagar, Shalabh ; Amrutur, Bharadwaj ; Ghosal, Ashitava ; Kolathaya, Shishir (2019) Learning Active Spine Behaviors for Dynamic and Efficient Locomotion in Quadruped Robots In: 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 14-18 Oct. 2019, New Delhi, India.

Kamanchi, Chandramouli ; Diddigi, Raghuram Bharadwaj ; Prabuchandran, K.J. ; Bhatnagar, Shalabh (2019) An Online Sample-Based Method for Mode Estimation Using ODE Analysis of Stochastic Approximation Algorithms IEEE Control Systems Letters, 3 (3). pp. 697-702. ISSN 2475-1456

Kamanchi, Chandramouli ; Diddigi, Raghuram Bharadwaj ; Prabuchandran, K.J. ; Bhatnagar, Shalabh (2019) An Online Sample-Based Method for Mode Estimation Using ODE Analysis of Stochastic Approximation Algorithms In: IEEE Conference on Decision and Control, Dec 11-13, 2019, Nice, France.

John, Indu ; Karumanchi, Ravikumar ; Bhatnagar, Shalabh (2019) Predictive and Prescriptive Analytics for Performance Optimization: Framework and a Case Study on a Large-Scale Enterprise System In: 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 16-19 Dec. 2019, Boca Raton, FL, USA.

Singla, Abhik ; Bhattacharya, Shounak ; Dholakiya, Dhaivat ; Bhatnagar, Shalabh ; Ghosal, Ashitava ; Amrutur, Bharadwaj ; Kolathaya, Shishir (2019) Realizing Learned Quadruped Locomotion Behaviors through Kinematic Motion Primitives In: International Conference on Robotics and Automation (ICRA), 20-24 May 2019, Montreal, QC, Canada.

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh (2019) Stability of Stochastic Approximations With “Controlled Markov” Noise and Temporal Difference Learning IEEE Transactions on Automatic Control, 64 (6). pp. 2614-2620. ISSN 0018-9286

Joseph, Ajin George ; Bhatnagar, Shalabh (2019) Stochastic Approximation Trackers for Model-Based Search In: 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 24-27 Sept. 2019, Monticello, IL, USA.

Kolathaya, Shishir ; Ghosal, Ashitava ; Amrutur, Bharadwaj ; Joglekar, Ashish ; Shetty, Suhan ; Dholakiya, Dhaivat ; Abhimanyu, . ; Sagi, Aditya ; Bhattacharya, Shounak ; Singla, Abhik ; Bhatnagar, Shalabh (2019) Trajectory based Deep Policy Search for Quadrupedal Walking In: 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 14-18 Oct. 2019, New Delhi, India.

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh (2018) Analysis of Gradient Descent Methods With Nondiminishing Bounded Errors IEEE Transactions on Automatic Control, 63 (5). pp. 1465-1471. ISSN 0018-9286

Chandramouli, K. ; Prabuchandran, K.J. ; Sai Koti Reddy, D. ; Bhatnagar, Shalabh (2018) Generalized Deterministic Perturbations For Stochastic Gradient Search In: IEEE Conference on Decision and Control (CDC), 17-19 Dec. 2018, Miami, FL, USA.

Zhou, Enlu ; Bhatnagar, Shalabh (2018) Gradient-Based Adaptive Stochastic Search for Simulation Optimization Over Continuous Space INFORMS Journal on Computing, 30 (1). pp. 154-167. ISSN 1091-9856

Lakshminarayanan, Chandrashekar ; Bhatnagar, Shalabh ; Szepesvari, Csaba (2018) A Linearly Relaxed Approximate Linear Program for Markov Decision Processes IEEE Transactions on Automatic Control, 63 (4). pp. 1185-1191. ISSN 0018-9286

Diddigi, Raghuram Bharadwaj ; Prabuchandran, K.J. ; Bhatnagar, Shalabh (2018) Novel Sensor Scheduling Scheme for Intruder Tracking in Energy Efficient Sensor Networks IEEE Wireless Communications Letters, 7 (5). pp. 712-715. ISSN 2162-2337

Yaji, Vinayaka G. ; Bhatnagar, Shalabh (2018) Stochastic recursive inclusions with non-additive iterate-dependent Markov noise Stochastics, 90 (3). pp. 330-363. ISSN 1744-2508

Karmakar, Prasenjit ; Bhatnagar, Shalabh (2018) Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning Mathematics of Operations Research, 43 (1). pp. 130-151. ISSN 0364-765X

Joseph, Ajin George ; Bhatnagar, Shalabh (2018) An incremental off-policy search in a model-free Markov decision process using a single sample path Machine Learning, 107 (6). pp. 969-1011. ISSN 0885-6125

Joseph, Ajin George ; Bhatnagar, Shalabh (2018) An online prediction algorithm for reinforcement learning with linear function approximation using cross entropy method Machine Learning, 107 (8-10). pp. 1385-1429. ISSN 0885-6125

Bhatnagar, Shalabh ; Patel, Sanjeev ; Karmeshu, . (2018) A stochastic approximation approach to active queue management Telecommunication Systems, 68 (1). pp. 89-104. ISSN 1018-4864

Bharadwaj, D. Raghuram ; Reddy, D. Sai Koti ; Bhatnagar, Shalabh ; Narayanam, Krishnasuri (2018) A unified decision making framework for supply and demand management in microgrid networks In: IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), 29-31 Oct. 2018, Aalborg, Denmark.

L. A., Prashanth ; Bhatnagar, Shalabh ; Fu, Michael ; Marcus, Steve (2017) Adaptive System Optimization Using Random Directions Stochastic Approximation IEEE Transactions on Automatic Control, 62 (5). pp. 2223-2238. ISSN 0018-9286

Karmeshu, . ; Patel, Sanjeev ; Bhatnagar, Shalabh (2017) Adaptive mean queue size and its rate of change: queue management with random dropping Telecommunication Systems, 65 (2). pp. 281-295. ISSN 1018-4864

Joseph, Ajin George ; Bhatnagar, Shalabh (2017) Bounds for off-policy prediction in reinforcement learning In: International Joint Conference on Neural Networks (IJCNN), 14-19 May 2017, Anchorage, AK.

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh (2017) A Generalization of the Borkar-Meyn Theorem for Stochastic Recursive Inclusions Mathematics of Operations Research, 42 (3). pp. 648-661. ISSN 0364-765X

Joseph, Ajin George ; Bhatnagar, Shalabh (2017) An Incremental Fast Policy Search Using a Single Sample Path Part of the Lecture Notes in Computer Science book series (LNCS, volume 10597), 10597 . Springer Nature, pp. 3-10. ISBN 978-3-319-69899-1

Lakshmanan, K. ; Bhatnagar, Shalabh (2017) Quasi-Newton smoothed functional algorithms for unconstrained and constrained simulation optimization Computational Optimization and Applications, 66 (3). pp. 533-556. ISSN 0926-6003

Kumar, Sandeep ; Padakandla, Sindhu ; Chandrashekar, L. ; Parihar, Priyank ; Gopinath, K. ; Bhatnagar, Shalabh (2017) Scalable Performance Tuning of Hadoop MapReduce: A Noisy Gradient Approach In: IEEE 10th International Conference on Cloud Computing (CLOUD), 25-30 June 2017, Honololu, HI, USA.

Joseph, Ajin George ; Bhatnagar, Shalabh (2017) A model based search method for prediction in model-free Markov decision process In: International Joint Conference on Neural Networks (IJCNN), 14-19 May 2017, Anchorage, AK.

Lakshminarayanan, Chandrashekar ; Bhatnagar, Shalabh (2017) A stability criterion for two timescale stochastic approximation schemes Automatica, 79 . pp. 108-114. ISSN 0005-1098

Prabuchandran, K. J. ; Bhatnagar, Shalabh ; Borkar, Vivek S. (2016) Actor-Critic Algorithms with Online Feature Adaptation ACM Transactions on Modeling and Computer Simulation, 26 (4). pp. 1-26. ISSN 1049-3301

Reddy, D. Sai Koti ; Prashanth, L.A. ; Bhatnagar, Shalabh (2016) Improved Hessian estimation for adaptive random directions stochastic approximation In: IEEE 55th Conference on Decision and Control (CDC), 12-14 Dec. 2016, Las Vegas, NV, USA.

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2016) Multi-armed bandits based on a variant of Simulated Annealing Indian Journal of Pure and Applied Mathematics, 47 (2). pp. 195-212. ISSN 0019-5588

Bhatnagar, Shalabh ; Lakshmanan, K. (2016) Multiscale Q-learning with linear function approximation Discrete Event Dynamic Systems, 26 (3). pp. 477-509. ISSN 0924-6703

B. N., Ranganath ; Bhatnagar, Shalabh (2016) Scalable focussed entity resolution In: International Joint Conference on Neural Networks (IJCNN), 24-29 July 2016, Vancouver, BC, Canada.

Ramaswamy, Arunselvan ; Bhatnagar, Shalabh (2016) Stochastic recursive inclusion in two timescales with an application to the Lagrangian dual problem Stochastics, 88 (8). pp. 1173-1187. ISSN 1744-2508

L.A., Prashanth ; H.L., Prasad ; Bhatnagar, Shalabh ; Chandra, Prakash (2016) A constrained optimization perspective on actor–critic algorithms and application to network routing Systems & Control Letters, 92 . pp. 46-51. ISSN 01676911

Joseph, Ajin George ; Bhatnagar, Shalabh (2016) A randomized algorithm for continuous optimization In: Winter Simulation Conference (WSC), 11-14 Dec. 2016, Washington, DC, USA.

Padakandla, Sindhu ; Prabuchandran, K.J. ; Bhatnagar, Shalabh (2015) Energy Sharing for Multiple Sensor Nodes With Finite Buffers IEEE Transactions on Communications, 63 (5). pp. 1811-1823. ISSN 0090-6778

Lakshminarayanan, Chandrashekar ; Bhatnagar, Shalabh (2015) A Generalized Reduced Linear Program for Markov Decision Processes In: Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), Jan 25-30, Austin, Texas, USA.

Yaji, Vinayaka G. ; Bhatnagar, Shalabh (2015) Necessary and sufficient conditions for optimality in constrained general sum stochastic games Systems & Control Letters, 85 . pp. 8-15. ISSN 01676911

Bhatnagar, Shalabh ; Prashanth, L. A. (2015) Simultaneous Perturbation Newton Algorithms for Simulation Optimization Journal of Optimization Theory and Applications, 164 (2). pp. 621-643. ISSN 0022-3239

Prashanth, L A ; Prasad, H L ; Desai, Nirmit ; Bhatnagar, Shalabh ; Dasgupta, Gargi (2015) Simultaneous perturbation methods for adaptive labor staffing in service systems Simulation, 91 (5). pp. 432-455. ISSN 0037-5497

Joseph, Ajin George ; Bhatnagar, Shalabh (2015) A Stochastic Approximation Algorithm for Quantile Estimation In: Proceedings of 22nd International Conference on Neural Information Processing (ICONIP), Nov.9-12, 2015, Istanbul, Turkey.

H.L., Prasad ; L.A., Prashanth ; Bhatnagar, Shalabh (2015) Two-timescale Algorithms For Learning Nash Equilibria In General-sum Stochastic Games In: Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), May 4-8, 2015, Istanbul, Turkey.

Prashanth, L A ; Chatterjee, Abhranil ; Bhatnagar, Shalabh (2014) Adaptive sleep-wake control using reinforcement learning in sensor networks In: Sixth International Conference on Communication Systems and Networks (COMSNETS), 6-10 Jan. 2014, Bangalore, India.

Lakshminarayanan, Chandrashekar ; Dubey, Ayush ; Bhatnagar, Shalabh ; Balamurugan, Chithralekha (2014) A Markov Decision Process Framework For Predictable Job Completion Times On Crowdsourcing Platforms In: Proceedings of HCOMP, Nov. 2-4, 2014, Pittsburgh.

K.J., Prabuchandran ; A.N, Hemanth Kumar ; Bhatnagar, Shalabh (2014) Multi-agent reinforcement learning for traffic signal control In: 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), 8-11 Oct. 2014, Qingdao, China.

Ghoshdastidar, Debarghya ; Dukkipati, Ambedkar ; Bhatnagar, Shalabh (2014) Newton-based stochastic optimization using $q$-Gaussian smoothed functional algorithms Automatica, 50 (10). pp. 2606-2614. ISSN 0005-1098

Zhou, Enlu ; Bhatnagar, Shalabh ; Chen, Xi (2014) Simulation optimization via gradient-based stochastic search In: Proceedings of the Winter Simulation Conference 2014, 7-10 Dec. 2014, Savannah, GA, USA.

Ghoshdastidar, Debarghya ; Dukkipati, Ambedkar ; Bhatnagar, Shalabh (2014) Smoothed Functional Algorithms for Stochastic Optimization Using q -Gaussian Distributions ACM Transactions on Modeling and Computer Simulation, 24 (3). pp. 1-26. ISSN 1049-3301

Prashanth, L. A. ; Chatterjee, Abhranil ; Bhatnagar, Shalabh (2014) Two timescale convergent Q-learning for sleep-scheduling in wireless sensor networks Wireless Networks, 20 (8). pp. 2589-2604. ISSN 1022-0038

Yao, Hengshuai ; Szepesvari, Csaba ; Sutton, Rich ; Modayil, Joseph ; Bhatnagar, Shalabh (2014) Universal Option Models In: Advances in Neural Information processing Systems (NIPS), Dec. 8-11, Montreal, Canada.

K.J., Prabuchandran ; Bhatnagar, Shalabh ; Borkar, Vivek S. (2014) An actor critic algorithm based on Grassmanian search In: 53rd IEEE Conference on Decision and Control, 15-17 Dec. 2014, Los Angeles, CA, USA.

Bhatnagar, Shalabh ; Borkar, Vivek S. ; Prashanth, L. A. (2013) Adaptive Feature Pursuit: Online Adaptation of Features in Reinforcement Learning Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, 23 . John Wiley & Sons, Inc., pp. 517-534. ISBN 9781118453988

Prasad, H. L. ; Prashanth, L. A. ; Bhatnagar, Shalabh ; Desai, Nirmit (2013) Adaptive Smoothed Functional Algorithms for Optimal Staffing Levels in Service Systems Service Science, 5 (1). pp. 29-55. ISSN 2164-3962

Bhatnagar, Shalabh ; Borkar, Vivek S. ; K. J., Prabuchandran (2013) Feature Search in the Grassmanian in Online Reinforcement Learning IEEE Journal of Selected Topics in Signal Processing, 7 (5). pp. 746-758. ISSN 1932-4553

Ananthapadmanabharao, Prashanth Lakshmanra ; Horabailu, Laxminarayana Prasad ; Desai, Nirmit ; Bhatnagar, Shalabh (2013) Mechanisms for hostile agents with capacity constraints In: Proceedings of Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS2013), May 6-10, Saint Paul, Minnesota.

Prabuchandran, K. J. ; Meena, Sunil Kumar ; Bhatnagar, Shalabh (2013) Q-Learning Based Energy Management Policies for a Single Sensor Node with Finite Buffer IEEE Wireless Communications Letters, 2 (1). pp. 82-85. ISSN 2162-2337

Bhatnagar, S. ; Prasad, H.L. ; Prashanth, L.A. (2013) Stochastic Recursive Algorithms for Optimization Lecture Notes in Control and Information Sciences Series, 434 (1). Springer Nature. ISBN 978-1-4471-4284-3

Chakravarty, Saswata ; Padakandla, Sindhu ; Bhatnagar, Shalabh (2013) A simulation-based algorithm for optimal pricing policy under demand uncertainty International Transactions in Operational Research, 21 (5). pp. 737-760. ISSN 0969-6016

Prasad, H.L. ; Bhatnagar, S. (2012) General-sum stochastic games: Verifiability conditions for Nash equilibria Automatica, 48 (11). pp. 2923-2930. ISSN 0005-1098

Bhatnagar, Shalabh ; Lakshmanan, K. (2012) An Online Actor–Critic Algorithm with Function Approximation for Constrained Markov Decision Processes Journal of Optimization Theory and Applications, 153 (3). pp. 688-708. ISSN 0022-3239

Vemu, Koteswara Rao ; Bhatnagar, Shalabh ; Hemachandra, N. (2012) Optimal multi-layered congestion based pricing schemes for enhanced QoS Computer Networks, 56 (4). pp. 1249-1262. ISSN 1389-1286

Prashanth, L. A. ; Bhatnagar, S. (2012) Threshold Tuning Using Stochastic Optimization for Graded Signal Control IEEE Transactions on Vehicular Technology, 61 (9). pp. 3865-3880. ISSN 0018-9545

Lakshmanan, K. ; Bhatnagar, Shalabh (2012) A novel Q-learning algorithm with function approximation for constrained Markov decision processes In: 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 1-5 Oct. 2012, Monticello, IL, USA.

Ghoshdastidar, Debarghya ; Dukkipati, Ambedkar ; Bhatnagar, Shalabh (2012) q-Gaussian based Smoothed Functional algorithms for stochastic optimization In: IEEE International Symposium on Information Theory Proceedings, 1-6 July 2012, Cambridge, MA, USA.

Bhatnagar, Shalabh (2011) The Borkar–Meyn theorem for asynchronous stochastic approximations Systems & Control Letters, 60 (7). pp. 472-478. ISSN 0167-6911

Bhatnagar, Shalabh ; Karmeshu, . (2011) Monte-Carlo estimation of time-dependent statistical characteristics of random dynamical systems Applied Mathematical Modelling, 35 (6). pp. 3063-3079. ISSN 0307-904X

Karmeshu, . ; Bhatnagar, Shalabh ; Mishra, Vivek Kumar (2011) An Optimized SDE Model for Slotted Aloha IEEE Transactions on Communications, 59 (6). pp. 1502-1508. ISSN 0090-6778

LA, Prashanth ; Bhatnagar, Shalabh (2011) Reinforcement Learning With Function Approximation for Traffic Signal Control IEEE Transactions on Intelligent Transportation Systems, 12 (2). pp. 412-421. ISSN 1524-9050

Prashanth, L A ; Bhatnagar, Shalabh (2011) Reinforcement learning with average cost for adaptive control of traffic lights at intersections In: 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), 5-7 Oct. 2011, Washington, DC, USA.

Bhatnagar, Shalabh (2011) Simultaneous Perturbation and Finite Difference Methods John Wiley & Sons, Inc.. ISBN 9780470400630

Lakshmanan, K. ; Bhatnagar, Shalabh (2011) Smoothed Functional and Quasi-Newton Algorithms for Routing in Multi-stage Queueing Network with Constraints In: Proceedings of ICDCIT (Distributed Computing and Internet Technology, Lecture Notes in Computer Science, Feb 9-12, Bhubaneswar, India.

Bhatnagar, Shalabh ; Mishra, Vivek Kumar ; Hemachandra, Nandyala (2011) Stochastic Algorithms for Discrete Parameter Simulation Optimization IEEE Transactions on Automation Science and Engineering, 8 (4). pp. 780-793. ISSN 1545-5955

Prashanth, L. A. ; Prasad, H. L. ; Desai, Nirmit ; Bhatnagar, Shalabh ; Dasgupta, Gargi (2011) Stochastic Optimization for Adaptive Labor Staffing in Service Systems In: Proceedings of 9th International Conference on Service Oriented Computing (ICSOC), Dec 5-8, Cyprus.

Bhatnagar, Shalabh ; Hemachandra, N. ; Mishra, Vivek Kumar (2011) Stochastic approximation algorithms for constrained optimization via simulation ACM Transactions on Modeling and Computer Simulation, 21 (3). pp. 1-22. ISSN 1049-3301

Chakraborty, Anshuk ; Bhatnagar, Shalabh (2010) Optimized Policies for the Retransmission Probabilities in Slotted Aloha Simulation, 86 (4). pp. 247-261. ISSN 0037-5497

Reza Maei, Hamid ; Szepesv´ari, Csaba ; Bhatnagar, Shalabh ; Sutton, Richard S. (2010) Toward Off-Policy Learning Control with Function Approximation In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel.

Bhatnagar, Shalabh (2010) An actor–critic algorithm with function approximation for discounted cost constrained Markov decision processes Systems & Control Letters, 59 (12). pp. 760-766. ISSN 0167-6911

Ramana Reddy, G. ; Bhatnagar, Shalabh ; Rakesh, V. ; Chaturvedi, Vijay Prakash (2010) An efficient algorithm for scheduling in bluetooth piconets and scatternets Wireless Networks, 16 (7). pp. 1799-1816. ISSN 1022-0038

Kolavali, Sudha Rani ; Bhatnagar, Shalabh (2009) Ant Colony Optimization Algorithms for Shortest Path Problems In: Second Workshop on Network Control and Optimization (NET-COOP), September 8-10, 2008, Paris, France.

Bhatnagar, Shalabh ; Precup, Doina ; Silver, David ; Sutton, Richard S. ; Maei, Hamid ; Szepesvári, Csaba (2009) Convergent Temporal-difference Learning With Arbitrary Smooth Function Approximation In: NIPS'09: Proceedings of the 22nd International Conference on Neural Information Processing Systems, December 2009.

Sutton, Richard S. ; Maei, Hamid Reza ; Precup, Doina ; Bhatnagar, Shalabh ; Silver, David ; Szepesvári, Csaba ; Wiewiora, Eric (2009) Fast gradient-descent methods for temporal-difference learning with linear function approximation In: 26th International Conference on Machine Learning, June 14-18, 2009, Montreal, Canada.

Yao, Hengshuai ; Bhatnagar, Shalabh ; Szepesvari, Csaba (2009) LMS-2: Towards an algorithm that is as cheap as LMS and almost as efficient as RLS In: Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, 15-18 Dec. 2009, Shanghai, China.

Yao, Hengshuai ; Bhatnagar, Shalabh ; Diao, Dongcui ; Sutton, Richard S ; Szepesv\'ari, Csaba (2009) Multi-Step Dyna Planning for Policy Evaluation and Control In: Advances in Neural Information Processing Systems, Dec.7-11, Bangalore, India.

Bhatnagar, Shalabh ; Sutton, Richard S. ; Ghavamzadeh, Mohammad ; Lee, Mark (2009) Natural actor–critic algorithms Automatica, 45 (11). pp. 2471-2482. ISSN 0005-1098

Bhatnagar, Shalabh ; Sutton, Richard S. ; Ghavamzadeh, Mohammad ; Lee, Mark (2009) Natural actor–critic algorithms Automatica, 45 (11). pp. 2471-2482. ISSN 0005-1098

Bhatnagar, Shalabh ; Karmeshu, . ; Mishra, Vivek Kumar (2009) Optimal parameter trajectory estimation in parameterized SDEs ACM Transactions on Modeling and Computer Simulation, 19 (2). pp. 1-27. ISSN 1049-3301

Viswanath, P. ; Murty, Narasimha M. ; Bhatnagar, Shalabh (2009) Pattern Synthesis for Nonparametric Pattern Recognition Encyclopedia of Data Warehousing and Mining, Second Edition . IGI Global, pp. 1511-1516. ISBN 9781605660103

Yao, Hengshuai ; Bhatnagar, Shalabh ; Szepesv´ari, Csaba (2009) Temporal Difference Learning by Direct Preconditioning In: Multidisciplinary Symposium on Reinforcement Learning (MSRL), June 18-19, 2009, Montreal, Canada.

Patro, Rajesh Kumar ; Bhatnagar, Shalabh (2009) A probabilistic constrained nonlinear optimization framework to optimize RED parameters Performance Evaluation, 66 (2). pp. 81-104. ISSN 0166-5316

Bhatnagar, Shalabh ; Patro, Rajesh (2009) A proof of convergence of the B-RED and P-RED algorithms for random early detection IEEE Communications Letters, 13 (10). pp. 809-811. ISSN 1089-7798

Bhatnagar, Shalabh ; Babu, K. Mohan (2008) New algorithms of the Q-learning type Automatica, 44 (4). pp. 1111-1119. ISSN 0005-1098

Patro, Rajesh Kumar ; Bhatnagar, Shalabh (2008) An Optimal RIO With Statistical Delay Assurances In: National Conference on Communications (NCC), February 2-3, 2008, Mumbai, India.

Velusamy, Sudha ; Bhatnagar, Shalabh ; Basavaraja, S.V. ; Sridhar, V. (2008) SPSA based feature relevance estimation for video retrieval In: IEEE 10th Workshop on Multimedia Signal Processing, 8-10 Oct. 2008, Cairns, QLD, Australia.

Bhatnagar, Shalabh ; Abdulla, Mohammed Shahid (2008) Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes Simulation, 84 (12). pp. 577-600. ISSN 0037-5497

Bhatnagar, Shalabh ; Abdulla, Mohammed Shahid (2008) Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes Simulation, 84 (12). pp. 577-600. ISSN 0037-5497

Velusamy, Sudha ; Gopal, Lakshmi ; Bhatnagar, Shalabh ; Varadarajan, Sridhar (2008) An efficient ad recommendation system for TV programs Multimedia Systems, 14 (2). pp. 73-87. ISSN 0942-4962

Reddy, G. Ramana ; Bhatnagar, Shalabh (2008) An efficient and optimized bluetooth scheduling algorithm for scatternets In: 2nd International Symposium on Advanced Networks and Telecommunication Systems, 15-17 Dec. 2008, Mumbai, India.

Vignat, C. ; Bhatnagar, S. (2008) An extension of Wick’s theorem Statistics & Probability Letters, 78 (15). pp. 2404-2407. ISSN 0167-7152

Bhatnagar, Shalabh (2007) Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization ACM Transactions on Modeling and Computer Simulation, 18 (1). pp. 1-35. ISSN 1049-3301

Mishra, Vivek ; Bhatnagar, Shalabh ; Hemachandra, N. (2007) Discrete parameter simulation optimization algorithms with applications to admission control with dependent service times In: 46th IEEE Conference on Decision and Control, 12-14 Dec. 2007, New Orleans, LA, USA.

Chaturvedi, Vijay Prakash ; Rakesh, V. ; Bhatnagar, Shalabh (2007) An Efficient and Optimized Bluetooth Scheduling Algorithm for Piconets In: International Conference on Distributed Computing and Internet Technology, December 17-20, 2007, Bangalore, India.

Velusamy, Sudha ; Gopal, Lakshmi ; Varatharajan, Sridhar ; Bhatnagar, Shalabh (2007) Fuzzy Clustering Based Ad Recommendation for TV Programs European Conference on Interactive Television Interactive TV: A Shared Experience, Eds., 4471 . Springer Nature, pp. 175-184. ISBN 978-3-540-72559-6

Dukkipati, Ambedkar ; Bhatnagar, Shalabh ; Narasimha Murty, M. (2007) Gelfand–Yaglom–Perez theorem for generalized relative entropy functionals Information Sciences, 177 (24). pp. 5707-5714. ISSN 0020-0255

Vemu, K.R. ; Bhatnagar, S. ; Hemachandra, N. (2007) Link Route Pricing For Enhanced Qos Technical Report. Institute of Electrical and Electronics Engineers, New Orleans, LA, USA.

Vemu, Koteswara Rao ; Bhatnagar, Shalabh ; Hemachandra, N. (2007) Link route pricing for enhanced QoS In: 46th IEEE Conference on Decision and Control, 12-14 Dec. 2007, New Orleans, LA, USA.

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2007) Network flow-control using asynchronous stochastic approximation In: 46th IEEE Conference on Decision and Control, 12-14 Dec. 2007, New Orleans, LA, USA.

Dukkipati, Ambedkar ; Bhatnagar, Shalabh ; Murty, M. Narasimha (2007) On measure-theoretic aspects of nonextensive entropy functionals and corresponding maximum entropy prescriptions Physica A: Statistical Mechanics and its Applications, 384 (2). pp. 758-774. ISSN 0378-4371

Vemu, Koteswara Rao ; Bhatnagar, Shalabh ; Hemachandra, N. (2007) An Optimal Weighted-Average Congestion Based Pricing Scheme for Enhanced QoS In: International Conference on Distributed Computing and Internet Technology, December 17-20, 2007, Bangalore, India.

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2007) Parametrized Actor-Critic Algorithms for Finite-Horizon MDPs In: American Control Conference, 9-13 July 2007, New York, NY, USA.

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2007) Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes Discrete Event Dynamic Systems, 17 (1). pp. 23-52. ISSN 0924-6703

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2007) Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes Discrete Event Dynamic Systems, 17 (1). pp. 23-52. ISSN 0924-6703

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2007) Solving MDPs using Two-timescale Simulated Annealing with Multiplicative Weights In: American Control Conference, 9-13 July 2007, New York, NY, USA.

Bhatnagar, Shalabh ; Panigrahi, J. Ranjan (2006) Actor-critic algorithms for hierarchical Markov decision processes Automatica, 42 (4). pp. 637-644. ISSN 0005-1098

Sharma, Diksha ; Bhatnagar, Shalabh ; Chakraborty, Shyam (2006) An Algorithm For Dynamic Optimal Bandwidth Allocation In Communication Networks In: Fifth Asia Pacific International Symposium on Information Technology (APIS5), 2006, Hangzhou, China.

Patro, Rajesh Kumar ; Bhatnagar, Shalabh (2006) A Four-Timescale Algorithm for Constrained Stochastic Optimization of RED In: 45th IEEE Conference on Decision and Control, 13-15 Dec. 2006, San Diego, CA, USA.

Dukkipati, Ambedkar ; Murty, M. Narasimha ; Bhatnagar, Shalabh (2006) Nonextensive triangle equality and other properties of Tsallis relative-entropy minimization Physica A: Statistical Mechanics and its Applications, 361 (1). pp. 124-138. ISSN 0378-4371

Viswanath, P. ; Murty, M. Narasimha ; Bhatnagar, Shalabh (2006) Partition based pattern synthesis technique with efficient algorithms for nearest neighbor classification Pattern Recognition Letters, 27 (14). pp. 1714-1724. ISSN 0167-8655

Vaidya, Rahul ; Bhatnagar, Shalabh (2006) Robust optimization of Random Early Detection Telecommunication Systems, 33 (4). pp. 291-316. ISSN 1018-4864

Chinthalapati, V. L. Raju ; Bhatnagar, S. (2006) A Simultaneous Deterministic Perturbation Actor-Critic Algorithm with an Application to Optimal Mortgage Refinancing In: 45th IEEE Conference on Decision and Control, 13-15 Dec. 2006, San Diego, CA, USA.

Bhatnagar, Shalabh (2005) Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization ACM Transactions on Modeling and Computer Simulation, 15 (1). pp. 74-107. ISSN 1049-3301

Bhatnagar, Shalabh ; Kowshik, Hemant J. (2005) A Discrete Parameter Stochastic Approximation Algorithm for Simulation Optimization Simulation, 81 (11). pp. 757-772. ISSN 0037-5497

Dukkipati, A. ; Murty, M.N. ; Bhatnagar, S. (2005) Information theoretic justification of Boltzmann selection and its generalization to Tsallis case In: IEEE Congress on Evolutionary Computation, 2-5 Sept. 2005, Edinburgh, UK.

Bhatnagar, Shalabh ; Reddy, I. Bala Bhaskar (2005) Optimal Threshold Policies for Admission Control in Communication Networks via Discrete Parameter Stochastic Approximation Telecommunication Systems, 29 (1). pp. 9-31. ISSN 1018-4864

Viswanath, P. ; Murty, Narasimha ; Bhatnagar, Shalabh (2005) Overlap pattern synthesis with an efficient nearest neighbor classifier Pattern Recognition, 38 (8). pp. 1187-1195. ISSN 0031-3203

Viswanath, P. ; Murty, M. Narasimha ; Bhatnagar, Shalabh (2005) Pattern Synthesis for Large-Scale Pattern Recognition Encyclopedia of Data Warehousing and Mining . IGI Global, pp. 902-905. ISBN 9781591405573

Dukkipati, Ambedkar ; Murty, M. Narasimha ; Bhatnagar, Shalabh (2005) Properties of Kullback-Leibler cross-entropy minimization in nonextensive framework In: International Symposium on Information Theory, 2005. ISIT 2005., 4-9 Sept. 2005, Adelaide, SA, Australia.

Abdulla, Mohammed Shahid ; Bhatnagar, Shalabh (2005) Solution of Mdps Using Simulation-Based Value Iteration In: Second IFIP Conference on Artificial Intelligence Applications and Innovations, 2005, Beijing, China.

Bhatnagar, S. ; Kumar, S. (2005) A reinforcement learning based algorithm for markov decision processes In: International Conference on Intelligent Sensing and Information Processing, 4-7 Jan. 2005, Chennai, India.

Dukkipati, A. ; Narasimha Murty, M. ; Bhatnagar, S. (2004) Cauchy annealing schedule: an annealing schedule for Boltzmann selection scheme in evolutionary algorithms In: 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753), 19-23 June 2004, Portland, OR, USA.

Vaidya, R. ; Bhatnagar, S. (2004) Correlation based optimization of random early detection In: IEEE INDICON 2004. First India Annual Conference, 2004, 20-22 December 2004, Kharagpur, India.

Viswanath, P. ; Narasimha Murty, M. ; Bhatnagar, Shalabh (2004) Fusion of multiple approximate nearest neighbor classifiers for fast and efficient classification Information Fusion, 5 (4). pp. 239-250. ISSN 1566-2535

Panigrahi, J.R. ; Bhatnagar, S. (2004) Hierarchical decision making in semiconductor fabs using multi-time scale Markov decision processes In: 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601), 14-17 Dec. 2004, Nassau, Bahamas.

Vaidya, Rahul ; Bhatnagar, Shalabh (2004) Optimized RIO for DiffServ Networks In: Information and Computer Science (ICICS), 2004, Dhahran, Saudi Arabia.

Viswanath, P. ; Narasimha Murty, M. ; Bhatnagar, Shalabh (2004) A Pattern Synthesis Technique To Reduce The Curse Of Dimensionality Effect In: International Conference on Knowledge Based Computer Systems (KBCS), 2004, Hyderabad, India.

Bhatnagar, S. ; Kumar, S. (2004) A Simultaneous Perturbation Stochastic Approximation-Based Actor–Critic Algorithm for Markov Decision Processes IEEE Transactions on Automatic Control, 49 (4). pp. 592-598. ISSN 0018-9286

Viswanath, R. ; Narasimha Murty, M. ; Bhatnagar, S. (2004) A pattern synthesis technique with an efficient nearest neighbor classifier for binary pattern recognition In: 17th International Conference on Pattern Recognition, 2004. ICPR 2004., 26-26 Aug. 2004, Cambridge, UK.

Bhatnagar, Shalabh ; Borkar, Vivek S. (2003) Multiscale Chaotic SPSA and Smoothed Functional Algorithms for Simulation Optimization Simulation, 79 (10). pp. 568-580. ISSN 0037-5497

Dukkipati, A. ; Murty, M.N. ; Bhatnagar, S. (2003) Quotient evolutionary space: abstraction of evolutionary process w.r.t. macroscopic properties In: Congress on Evolutionary Computation, 2003. CEC '03., 8-12 Dec. 2003, Canberra, ACT, Australia.

Bhatnagar, Shalabh ; Fu, Michael C. ; Marcus, Steven I. ; Wang, I-Jeng (2003) Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences ACM Transactions on Modeling and Computer Simulation, 13 (2). pp. 180-209. ISSN 1049-3301

Cao, Xi-Ren ; Ren, Zhiyuan ; Bhatnagar, Shalabh ; Fu, Michael ; Marcus, Steven (2002) A time aggregation approach to Markov decision processes Automatica, 38 (6). pp. 929-943. ISSN 0005-1098

Bhatnagar, S. ; Fu, M.C. ; Marcus, S.I. ; Fard, P.J. (2001) Optimal structured feedback policies for ABR flow control using two-timescale SPSA IEEE/ACM Transactions on Networking, 9 (4). pp. 479-491. ISSN 1063-6692

Bhatnagar, Shalabh ; FU, Michael C. ; Marcus, Steven I. ; Bhatnagar, Shashank (2001) Two-timescale algorithms for simulation optimization of hidden Markov models IIE Transactions, 33 (3). pp. 245-258. ISSN 0740-817X

He, Ying ; Bhatnagar, Shalabh ; Fu, Michael C. ; Marcus, Steven I. (2000) Approximate Policy Iteration for Semiconductor Fab-Level Decision Making - a Case Study Institute for Systems Research Technical Reports .

Bhatnagar, Shashank ; Marcus, Steven I. ; Fu, Michael C. ; Bhatnagar, Shalabh (2000) Randomized Difference Two-Timescale Simultaneous Perturbation Stochastic Approximation Algorithms for Simulation Optimization of Hidden Markov Models Technical Report. Defense Technical Information Center.

Bhatnagar, S. ; Fernandez-Gaucherand, E. ; Fu, M.C. ; He, Ying ; Marcus, S.I. (1999) A Markov decision process model for capacity expansion and allocation In: 38th IEEE Conference on Decision and Control (Cat. No.99CH36304), 7-10 Dec. 1999, Phoenix, AZ, USA.

van der Mei, Robert D. ; Bhatnagar, Shalabh ; Fu, Michael C. ; Marcus, Steven I. ; Heyman, Daniel P. (1999) Rate-based ABR flow control using two timescale SPSA In: Proceedings of SPIE Conference on Performance and Control of Network Systems III, 18 August 1999, Boston, MA, United States.

Bhatnagar, S. ; Sharma, V. (1998) Optimal control of a feedback queue via stochastic approximation In: IEEE GLOBECOM 1998 (Cat. NO. 98CH36250), 8-12 Nov. 1998, Sydney, NSW, Australia.

Bhatnagar, Shalabh ; Borkar, Vivek S. (1998) A two Timescale Stochastic Approximation Scheme for Simulation-Based Parametric Optimization Probability in the Engineering and Informational Sciences, 12 (4). pp. 519-531. ISSN 0269-9648

Bhatnagar, Shalabh ; Borkar, Vivek S. (1997) Multiscale Stochastic Approximation for Parametric Optimization of Hidden Markov Models Probability in the Engineering and Informational Sciences, 11 (4). pp. 509-522. ISSN 0269-9648

Gupta, V.H. ; Bhatnagar, S. (1997) An optimal fuel-injection policy for performance enhancement in internal combustion engines Sadhana Academy Proceedings in Engineering Sciences, 22 (4). pp. 545-552. ISSN 0256-2499

Bhatnagar, Shalabh ; Borkar, Vivek S. (1995) A Convex Analytic Framework for Ergodic Control of Semi-Markov Processes Mathematics of Operations Research, 20 (4). pp. 923-936. ISSN 0364-765X

Bhatnagar, Shalabh ; Fu, Michael C. ; Marcus, Steven I. Optimal Multilevel Feedback Policies for ABR Flow Control using Two Timescale SPSA Technical Report. University of Maryland Libraries.