Flow Control as Stochastic Optimal Control Problem with Incomplete Information
The nonlinear stochastic control problem related with flow control is
considered. The state of the link is described by controlled hidden Markov
process while the loss flow is described by the counting process with the
intensity depending on the current transmission rate and unobserved link
state. The control is the transmission rate and have to be chosen as
non-anticipating process depending on the observation of the loss process. The
aim of the control is to achieve the maximum of some utility function taking
into account the losses of transmitted information. Originally the problem
belongs to a class of stochastic control with incomplete information, however,
the optimal filtering equations giving the estimation of the current link
state based on the observation of the loss process give the opportunity to
reduce the problem to the standard stochastic control problem with full
observations. Then, a necessary optimality condition is derived in the form of
stochastic maximum principle which allows us to obtain explicit analytic
expressions for the optimal control in some particular cases. The optimal and
suboptimal controls are investigated and compared with the flow control
schemes which is used in TCP/IP networks. In particular, the optimal control
demonstrates a much smoother behaviour than the currently used TCP/IP
congestion control.
Philippe Nain
Last modified: Tue Mar 15 14:32:26 MET 2005