Control of Markov chains: applications
We have been working mainly on applications of MDPs
in queueing systems. We also have a
survey on applications of MDPs to communication networks
(
postscript file
,
pdf file
),
that appeared in
Handbook of Markov Decision Processes Methods and Applications,
edited by E. A. Feinberg and A. Shwartz, Kluwer, 2001.
Quite often dynamic programming techniques can be
used in order to obtain the structure of optimal
policies in the control of queueing systems, even when
the state space is infinite. This
then allows to reduce a dynamic control problem into
an optimization problem over a relatively small parameter set,
and, in some cases, to fully solve the problem.
Our work that concerns applications of MDPs to queueing
network is mostly focused on establishing the structure
of optimal policies and values.
Papers:
E. Altman and A. Shwartz
,
Optimal priority assignment: a time sharing approach
,
IEEE - Transactions on Automatic Control
,
Vol.
34, No. 10, pp. 1098-1102, 1989.
E. Altman and
P. Nain
,
Optimal Control of an M/G/1 Queue with Vacations
,
IEEE - Transactions on Automatic Control
,
Vol.
38, No. 12, pp. 1766-1775, 1993.
E. Altman and
P. Nain
,
Optimality of a threshold policy in the M/M/1
queue with repeated vacations
,
ZOR - Mathematical Methods of Operations Research
,
Vol. 44, Issue 2, pp. 75-96, 1996.
E. Altman and H. J. Kushner
,
Admission Control for Combined Guaranteed Performance and Best
Effort Communications Systems Under Heavy Traffic
,
SIAM J. Control and Optimization
,
pp. 1780-1807, vol 37 no 6, 1999.
E. Altman and H. J. Kushner
,
Control of Polling in Presence of Vacations in Heavy Traffic
with Applications to Satellite and Mobile Radio Systems
,
to appear in
SIAM J. Control and Optimization
,
2001.
E. Altman,
G. Koole and
T. Jimenez
,
On optimal call admission control in a resource-sharing system
,
IEEE Trans. on Communications
,
Volume 49, Number 9, pp. 1659-1668, 2001.
Eitan Altman
,
Applications of Markov Decision Processes in Communication
Networks: a Survey
(
pdf
,
ps
),
in
Handbook of Markov Decision Processes Methods and Applications,
edited by E. A. Feinberg and A. Shwartz, Kluwer, 2001.
W. K. Ching, M. K. Ng, K. K. Wong and E. Altman,
,
Customer lifetime value: stochastic optimization approach
,
Journal of the Operational Research Society
,
Vol 55, pp 860-868, 2004.
Other control problems of queueing systems are described in
sections
Back to my
Web page
.