Non Markovian stochastic control

Together with Bruno Gaujal and Arie Hordijk, we layed some foundations for non Markovian control of discrete event dynamic systems in a series of papers. The theory is related to multimodularity properties of the immediate cost functions. As applications, we consider admission control, and control of polling, service and vacations, into quite general systems (those that can be described as linear in the max-plus algebra). We pursue our research in non-Markovian closed-loop control as well, and we currently study new applications of our theory.

Monograph:

E. Altman, B. Gaujal and A. Hordijk , Discrete-event control of stochastic networks: Multimodularity and Regularity (Copyrights: Springer Verlag) Springer Verlag, Series: Lecture Notes in Mathematics 2003, XIV, 313 p., Softcover ISBN: 3-540-20358-3.

Papers:

E. Altman, B. Gaujal and A. Hordijk , Multimodularity, convexity and optimization properties , Math. of Oper. Research, Volume 25, pp. 324-347, 2000.

E. Altman, B. Gaujal and A. Hordijk , Admission control in stochastic event graphs , IEEE Automatic Control, Vol. 45, No. 5, pp. 854-867, 2000.

E. Altman, B. Gaujal and A. Hordijk , Balanced sequences and optimal routing , JACM, Vol. 47, Issue 4, pp. 752-775, 2000.

E. Altman, B. Gaujal and A. Hordijk , Optimal Open-Loop Control of Vacations, Polling and Service Assignment , INRIA Report No. RR-3261, Sophia-Antipolis, France, June 1997. Submitted to QUESTA.
E. Altman, S. Bhulai, B. Gaujal and A. Hordijk , Optimal Routing Problems and Multimodularity , INRIA Report No. RR-3727, Sophia-Antipolis, France, June 1999. To appear in Journal of Applied Probability, Vol. 37, pp. 668-684, No. 3, Sept. 2000.

E. Altman, B. Gaujal, A. Hordijk and G. Koole , Optimal admission, routing and service assignment control: the case of single buffer queues , Proc. of the 37th IEEE Conference on Decision and Control}, Dec. '98, Tampa, Florida, USA.

E. Altman, B. Gaujal and A. Hordijk , Regularity for admission control comparisons , submitted. A shorter version appeard in Proc. of the 37th IEEE Conference on Decision and Control, Dec. '98, Tampa, Florida, USA.

E. Altman, B. Gaujal and A. Hordijk , Simplex convexity with application to open loop stochastic control in networks Proceedings of 39th IEEE CDC, pp. 1852-1857, Sydeney, Australia, Dec. 2000.

Another paper on non Markovian control is

E. Altman and A. Hordijk , Applications of Borovkov's Renovation Theory to Non-Stationary Stochastic Recursive Sequences and their Control , Advances of Applied Probability , Vol. 29, pp. 388-413, 1997.

Back to my Web page .