Stochastic Systems & Learning Laboratory (S2L2)

 

Research Statement


Due to significant change in the group’s focus over the last few years, the name has changed to Stochastic Systems and Learning Laboratory (S2L2, for short).


The main activities of the research lab are Stochastic Systems, Stochastic Optimization, Statistical Learning, Queueing Theory, Game Theory and Power System Economics. The application domains currently of interest are: Energy/Power systems, Healthcare operations, Transportation and Communication networks and systems.


My interests in Stochastic Systems span stochastic control theory, approximate dynamic programming and reinforcement learning. My group has developed, and is still developing `Empirical dynamic programming’ (EDP), or dynamic programming by simulation. This seems to be a very useful alternative to reinforcement learning algorithms.


My interests in Stochastic Optimization span primarily Risk-aware Stochastic Optimization. We have developed a mathematical framework and techniques that are remarkably effective in solving what are otherwise seemingly very difficult problems in risk-aware optimization.


My interests in Statistical Learning span PAC learning, online learning in multi-armed bandit models, Reinforcement Learning. I developed a PAC Theory for Markov Decision Processes. Recent work has included online decentralized learning algorithms for multi-player multi-armed bandit models.


My work on Queueing Theory has included developing non-classical models called `transitory queues’ and their theory via fluid and diffusion limits. These are relevant in `transitory’ situations wherein either only a finite population of users arrive, or the queues exist only for finite time.


My work on Game Theory has primarily revolved around Network Market Design, wherein we have developed theory and mechanisms for resource allocation in network and combinatorial settings. These are very relevant for bandwidth and spectrum allocation via auctions.


My recent interests include Power System Economics wherein I have been working on analysis of electricity markets, as well as pricing algorithms for demand response. I have also developed stochastic mechanism design for renewable energy integration.


I also have a burgeoning interest in Healthcare System Operations. This has revolved around Data Analytics for Healthcare, and Data-driven Operations Management.



Group Members and PhD Students


  1. 1.Wenyuan Tang (PhD Student, 5th year)

  2. 2.Naumaan Nayyar (PhD Student, 5th year)

  3. 3.Hiteshi Sharma (PhD student, 1st year)

  4. 4.Mehdi Jafarnia (PhD Student, 1st year)

  5. 5.Yunan Zhou (PhD student, 1st year)

  6. 6.Abhishek Gupta (Postdoc, 1st year)


Group Alumni


  1. 1.Srinivas Yerramalli (*co-advised PhD, March 2013). Currently: QualComm/IIIT Hyderabad

  2. 2.William Haskell (Postdoc, June 2014). Currently: National Univ. of Singapore (assistant prof.)

  3. 3.Harsha Honnappa (PhD, Dec. 2014). Currently: Purdue University (assistant prof., Jan 2015)

  4. 4.Dileep Kalathil (PhD, Oct. 2014). Currently: UC Berkeley (Postdoc)



Research Support and Funded Projects


  1. 1.John H. Zumberge Faculty Research and Innovation Award, 2009-10: “The economics of Quality of Service on the Internet”.

  2. 2.NSF Network Science and Engineering (NetSE) Grant, 2009-13 (PI: Rahul Jain): “NetSE: Small; Cooperation and incentives in communication and social networks”.

  3. 3.NSF CAREER Award, 2010-2015 (PI: Rahul Jain): “CAREER: Network Economics: Theory and architectures for incentive-engineered networks”.

  4. 4.Air Force Office of Scientific Research (AFOSR) Grant, 2010-2013 (PI: Rahul Jain): “Distributed control and information fusion over communication networks”.


  1. 5.IBM Faculty Award, 2010: “Smarter Cities: Distributed optimization and control framework for Smart energy networks”.


  1. 6.Office of Naval Research (ONR) Young Investigator Award (2012-2015): “Stochastic dynamic optimization and games: Simulation and learning methods”.



------------------------------------------------------------------------------------------

Publications by Area


Network Economics and Game Theory


(List is incomplete and not updated since mid-2012. Please see the publications page for recent and journal papers. Most papers can be downloaded here.)


Network Market Design & Power System Economics


  1. 1. W. Tang and R. Jain, “Auction designs for aggregating stochastic resources for smart-grid networks”, submitted to IEEE Control and Decision Conf. (CDC), March 2012.


  1. 2. Q. Wang, M. Liu and R. Jain, “Dynamic pricing of power in smart-grid networks”, submitted to IEEE Control and Decision Conf. (CDC), March 2012.


  1. 3. W. Tang and R. Jain, “Stochastic resource auctions for renewable energy integration”, The Allerton Conference on Communications, Computation and Controls, September 2011.


  1. 4. W. Tang and R. Jain, “Hierarchical auctions for network resource allocation”, Proc. of the Intl’ Conference on Game Theory in Networks (GameNets), April 2011.


  1. 5. R. Jain and J. Walrand, “An efficient Nash-implementation mechanism for divisible resource allocation”,  Automatica, 46(8):1276-1283, August 2010.


  1. 6. R. Jain and J. Walrand, “An efficient mechanism for network bandwidth auction”, IEEE NOMS Workshop on Bandwidth on Demand (BoD), April 2008.


  1. 7. R. Jain, “Designing a bipartite matching market”, Proc. IEEE Conf. on Decision and Control (CDC), December 2007. 


  1. 8. R. Jain, “Stability of marriage and dowry”, Intl. Conf. on Applied Mathematics, Plovdiv,  Bulgaria, August 2007. (non-archived)


  1. 9. R. Jain and P. Varaiya, “A design for an asymptotically efficient combinatorial Bayesian market: Generalizing the Satterthwaite-Williams mechanism”, Int. Conf. on Game Theory, Stony Brook, July 2007. (non-archived)


  1. 10. C. Kaskiris, R. Jain, R. Rajagopal and P. Varaiya, “Combinatorial Auction Bandwidth Trading: An Experimental Study”, in Developments in Experimental Economics, Springer Lecture Notes in Economics and Mathematical Systems, 2007.


  1. 11. A. Dimakis, R. Jain and J. Walrand, “Mechanisms for efficient allocation in divisible capacity networks”, Proc. IEEE Conf. Decision and Control (CDC), December 2006. 


  1. 12. C. Kaskiris, Y. Butun and R. Jain, “An experimental analysis of a combinatorial market mechanism for bandwidth trading”, Proc. of the IEEE Workshop on Bandwidth on Demand (BoD), November 2006. 


  1. 13. R. Jain and P. Varaiya, “Efficient market mechanisms for network resource allocation”,  Proc. IEEE Conf. Decision and Control (CDC), December 2005. (invited paper) 


  1. 14. R. Jain, C. Kaskiris, R. Rajagopal and P. Varaiya, “Combinatorial auction design for bandwidth trading: An experimental study”, International Conf. on Experiments in Economic Sciences (EES), December 2004.


  1. 15. R. Jain and P. Varaiya, “An efficient incentive-compatible combinatorial market mechanism”, (invited paper) Proc. of the Allerton Conference on Systems, Communications and Controls, October 2004.


  1. 16. R. Jain and P.P.Varaiya, “Combinatorial exchange mechanisms for efficient bandwidth allocation”, Communications in Information and Systems, 3(4):305-324, September 2004.


  1. 17. R. Jain, A. Dimakis and P. Varaiya, “On the existence of competitive equilibria in bandwidth markets”, Proc. of the Allerton Conference on Systems, Communications and Controls, October 2002.



Queueing and Games


  1. 1.H. Honnappa, R. Jain and A. Ward, “A transitory queueing model, and its process limits”, submitted to The ValueTools Conference, May 2012.


  1. 2.H. Honnappa and R. Jain, “The concert queueing game: The network case”, submitted to IEEE Control and Decision Conf. (CDC), March 2012.


  1. 3.H. Honnappa, R. Jain and A. Ward, “Strategically arriving users into queueing systems”, SIAM Conference on Control and its Applications, July 2011. (non-archived, see arxiv version)


  1. 4.R. Jain, S. Juneja and N. Shimkin, “The concert queueing game: To wait or to be late”, Discrete Event Dynamic Systems, 21(1):103-134, 2011.


  1. 5.H. Honnappa and R. Jain, “Strategic arrivals into queueing networks”, INFORMS Annual Meeting, Nov. 2010. (non-archived, see arxiv version)


  1. 6.H. Honnappa and R. Jain, “Strategic arrivals into queueing networks”, Proc. of the Allerton Conference on Systems, Communications and Control, September 2010.


  1. 7.R. Jain, S. Juneja and N. Shimkin, “Queueing for Timely Service: Equilibrium Analysis and Social Efficiency”,  INFORMS MSOM Conference, September 2010.


  1. 8.P. Dube and R. Jain, “DiffServ Pricing Games in Multi-class Queueing Network Models”, Proc. of the International Teletraffic Congress (ITC), September 2010.


  1. 9.R. Jain, S. Juneja and N. Shimkin, “The concert queueing game: To wait or to be late”,  presented at the Information Theory and Applications (ITA) Workshop,  January 2010.  (non-archived, see journal version)


  1. 10.R. Jain and Parijat Dube, “Bertrand games between multi-class queues”, Proc. of the IEEE Control and Decision Conference (CDC), December 2009. 


  1. 11.Sandeep Juneja and R. Jain, “The concert/cafetaria queueing problem: A game of arrivals”, Proc. of The ValueTools Conference, October 2009. (Best Paper Award at The ValueTools Conf.).


  1. 12.Sandeep Juneja and R. Jain, “The concert/cafetaria queueing problem: A game of arrivals”, INFORMS Applied Probability Society Conference, July 2009. (non-archived)


  1. 13.P. Dube and R. Jain, “Bertrand games between multi-class queues”, Proc. of the IEEE Control and Decision Conference (CDC), December 2009. 


  1. 14.R. Jain and Parijat Dube, “Queueing game models for differentiated services”, (invited) Proc. of the Intl’ Conference on Game Theory in Networks (GameNets), May 2009.


  1. 15.R. Jain and Parijat Dube, “N-player Bertrand and Cournot queueing games: Existence of equilibrium”, Proc. of the Allerton Conference on Systems, Communications and Control, September 2008.


  1. 16.P. Dube, R. Jain and C. Touati, “Pricing competition for queued services”, (invited) UCSD Information Theory and Applications (ITA) workshop, UCSD, January 2008. (non-archived)


  1. 17.P. Dube and R. Jain, “Pricing differentiated queued services”, INFORMS Annual Meeting, Seattle, November 2007. (non-archived)


  1. 18.P. Dube, R. Jain and C. Touati, “An analysis of oligopolistic competitive markets of queued services with QoS guarantees” INFORMS Applied Probability Soc. (APS) Meeting, July 2007. (non-archived)



Non-cooperative Multi-user Communications


  1. 1. S. Yerramalli, R. Jain and U. Mitra, “Characterization of equilibria for the degraded Gaussian broadcast channel”, submitted to IEEE Globecom, March 2012.


  1. 2. S. Yerramalli, R. Jain and U. Mitra, “Stability of Transmitter Cooperation Over a Multiple Access Channel”, to appear, Proc. Int. Conf. on Signal Proc. and Comm. (SPCOM), August 2012.


  1. 3. S. Yerramalli, R. Jain and U. Mitra, “A game theoretic model for the Gaussian broadcast channel”, to appear, IEEE Int. Symp. on Information Theory (ISIT), June 2012.


  1. 4. D. Kalathil and R. Jain, “Investigation for cooperative relaying in a simple information-theoretic model”, to appear, IEEE Int. Symp. on Information Theory (ISIT), June 2012.


  1. 5. S. Yerramalli, R. Jain and U. Mitra, “Coalition games for transmitter cooperation in wireless networks”, Proc. IEEE Int’l Symp. on Information Theory (ISIT), August 2011.


  1. 6. D. Kalathil and R. Jain, “Communication games on the generalized Gaussian relay channel”, Information Theory and Applications (ITA) Workshop, UCSD, Feb 2011.


  1. 7. D. Kalathil and R. Jain, “Communication games on the generalized Gaussian relay channel”, Proc. of the The Allerton Conference,  September 2010.


  1. 8. D. Kalathil and R. Jain, “A contracts-based approach to spectrum sharing in cognitive radio networks”, Proc. of the WiOpt Symposium, June 2010.


  1. 9. D. Kalathil and R. Jain, “Spectrum sharing through contracts”,  Proc. of IEEE DySpan Conference, April 2010.


Wireless Networks


  1. 1. M. Ergen, S. Coleri, B. Dundar, R. Jain, A. Puri, P. Varaiya, “Application of GPS to mobile IP and routing in wireless networks”, Proc. IEEE Vehicular Tech. Conf. (VTC), September 2002. 


  1. 2. R. Jain, “Geographical routing for wireless ad hoc networks”, IMA Workshop on Hot Topics in Wireless, University of Minnesota, Minneapolis, August 2001. (non-archived) 


  1. 3. R. Jain, “Geographical routing for wireless ad hoc networks using partial information”, DIMACS Workshop on The Foundations of Wireless Networks and Applications, Rutgers University and AT&T Shannon Labs, August 2000. (non-archived) 


  1. 4. R. Jain, B. Sadeghi and E.W. Knightly, “Towards coarse-grained mobile-QoS”, Proc. IEEE WOW-MOM, August 1999.


  1. 5. R. Jain and E.W. Knightly, “A framework for design and evaluation of mobile admission control algorithms”, Proc. of IEEE INFOCOM, March 1999.


  1. 6. R. Jain, A.Puri and R.Sengupta, “Geographical routing for wireless ad hoc networks using partial information”, IEEE Personal Communications, 8(1):48-57, February 2001.


  1. 7. R. Jain, “Key distribution and authentication protocol for secure wireless conferencing”, IETE Technical Review, 15(6):471-476, December 1998.



Stochastic Models: MDPs, Multi-Armed Bandits, and Stochastic Games


  1. 1.D. Kalathil, N, Nayyar and R. Jain, “Decentralized learning for multi-player multi-armed bandits”, submitted to IEEE Control and Decision Conf. (CDC), March 2012.


  1. 2.W. Haskell and R. Jain, “Dominance-constrained Markov decision processes”, submitted to IEEE Control and Decision Conf. (CDC), March 2012.


  1. 3.D. Kalathil, N, Nayyar and R. Jain, “Decentralized learning for multi-player multi-armed bandits”, UCSD Info. Theory and Appl. (ITA) Symp., Feb. 2012. (non-archived, see arxiv version)


  1. 4.Y. Gai, B. Krishnamachari and R. Jain, “Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations”, to appear, IEEE/ACM Trans. on Networking, 2012.


  1. 5.V. Borkar and R. Jain, “Risk-constrained Markov Decision Processes”, Proc. of the IEEE Control and Decision Conf. (CDC), December 2010.


  1. 6.R. Jain, “Solving Markov decision problems via simulations”, International Congress of Mathematicians (ICM) Satellite Conference on Probability and Stochastic Processes, August 2010. (non-archived)


  1. 7.Yi Gai, B. Krishnamachari and R. Jain, “Learning Multi-user channel allocations in cognitive radio networks: A combinatorial multi-armed bandit formulation”, Proc. of IEEE DySpan Conference, April 2010.


  1. 8.R. Jain and P. Varaiya, “Simulation-based optimization of Markov decision processes: An empirical process theory approach”, Automatica, 46(8):1297-1304, August 2010.


  1. 9.Vivek Borkar and R. Jain, “Risk-constrained Markov decision processes”, INFORMS Annual Meeting, November 2009. (non-archived)


  1. 10.Vivek Borkar and R. Jain, “Risk-constrained Markov decision processes”, INFORMS Applied Probability Society Conference, July 2009. (non-archived)


  1. 11.R. Jain and P. Varaiya, “PAC bounds for simulation-based optimization of Markov decision processes”, Proc. IEEE Conf. Decision and Control (CDC), December 2007. (invited SIAM paper)


  1. 12.R. Jain and P. Varaiya, “Simulation-based optimization for Markov decision processes and Multi-armed bandits: An empirical process theory approach”, Conf. on Stochastic Processes and Applications (SPA-07), Urbana-Champaign, IL, August 2007. (non-archived)


  1. 13.R. Jain and P.P. Varaiya, “Simulation-based uniform value function estimates of Markov decision processes”, SIAM J. Control and Optimization, 45(5):1633-1656, November 2006.


  1. 14.R. Jain and P.P. Varaiya, “Simulation-based uniform estimates of value functions of Markov decision processes”, Conf. Stochastic Processes and Applications (SPA-05), June 2005. (non-archived)


  1. 15.R. Jain and P.P. Varaiya, “Simulation-based uniform value estimates of discounted and average reward MDPs”, Proc. IEEE Conf. Decision and Control (CDC), December 2004.


  1. 16.R. Jain and P.P. Varaiya, “PAC learning for Markov decision processes and dynamic games”, Proc. IEEE Symp. on Information Theory (ISIT), June 2004.


  1. 17.R. Jain and P.P. Varaiya, “Extensions to PAC learning for partially observable Markov decision processes”, Proc. Conf. Information Systems and Sciences (CISS), March 2004.


Decentralized & Networked Control Systems


  1. 1.N. Naumaan, D. Kalathil and R. Jain, “Decentralized optimal control in asymmetric one-step delayed sharing patterns”, submitted to American Control Conf. (ACC), September 2012.


  1. 2.T. Simsek, R. Jain and P.P. Varaiya, “Scalar estimation and control with noisy binary observations”, IEEE Trans. Automatic Control, 49(9):1598-1603, September 2004.


  1. 3.R. Jain, T. Simsek and P.P.Varaiya, “Control under communication constraints”, Proc. IEEE Conf. Decision and Control (CDC), December 2002.



Disclaimer: The copyrights of the papers belong to the publishers. The papers are made available here for your convenience.