* Control of Markov chains: applications

We have been working mainly on applications of MDPs in queueing systems. We also have a survey on applications of MDPs to communication networks ( postscript file , pdf file ), that appeared in Handbook of Markov Decision Processes Methods and Applications, edited by E. A. Feinberg and A. Shwartz, Kluwer, 2001. Quite often dynamic programming techniques can be used in order to obtain the structure of optimal policies in the control of queueing systems, even when the state space is infinite. This then allows to reduce a dynamic control problem into an optimization problem over a relatively small parameter set, and, in some cases, to fully solve the problem. Our work that concerns applications of MDPs to queueing network is mostly focused on establishing the structure of optimal policies and values.


Other control problems of queueing systems are described in sections

Back to my Web page .