Asymptotic optimality of quantized approximations in stochastic control

Speaker: Serdar Yuksel, Dept. of Mathematics and Statistics, Queen's University, Canada

Quantization arises in stochastic control in the contexts of informational and computational constraints. On the informational side, this problem is relevant in 'control under information constraints' where the goal is to characterize jointly optimal coding and control policies under various performance criteria, such as some expected cost minimization or some stability criterion, when a controller has access to limited information. In stochastic control, quantization also arises in developing approximate representations of uncountable state/action space models with finite models, for discounted or average cost problems. Since for Markov Decision Processes with uncountable spaces the computation of optimal policies is known to be prohibitively hard, quantized models allow for tractable learning and computational algorithms. In this talk, we will present quite relaxed conditions under which finite models can be used to construct approximately optimal policies and, under further conditions, obtain explicit rates of convergence to the optimal cost of the original problem as the quantization rate increases. We consider various setups for the continuity conditions imposed on the transition kernels, the state and action spaces, and the partially observed case (where the relaxed nature of our assumptions finds particularly important applications). Our approximation results may be considered to be complete in that whenever we could state the existence of optimal policies, an approximation result follows; thus providing relaxed conditions compared with existing results in the literature. Using information theoretic tools, we show that the convergence rates are order-optimal for a further class of problems. Some examples will be provided. (Part of this talk is based on joint work with Naci Saldi and Tamas Linder).
Category Seminar
Location: EDIT Room 3364, Hörsalsvägen 9, Campus Johanneberg
Starts: 25 April, 2017, 14:00
Ends: 25 April, 2017, 15:00

Published: Fri 07 Apr 2017.