Seminar on Computational Learning and Adaptation

 
Kernel-Based Reinforcement Learning

Dirk Ormoneit
Department of Statistics
Stanford University
ormoneit@stat.stanford.edu

Kernel-based methods, which approach induction by placing weights on training cases, have attracted considerable attention in the machine learning and the statistics literatures. In this talk, we suggest a new application of kernel-smoothing to variants of reinforcement learning that attempt to estimate the value function for a continuous-state Markov decision process. Using standard methods, such as temporal-difference learning, one cannot show consistency, but, for the kernel-based method, we can derive the asymptotic distribution of the value-function estimate and establish rates of convergence. In closing, we consider an optimal investment problem that demonstrates the practical benefits of the kernel-based approach to reinforcement learning.

Joint work with Saunak Sen.

Return to the seminar schedule