Apprenticeship Learning with Applications in Autonomous Helicopter Flight and Robot Dogs

Many problems in control have unknown, stochastic, and highly non-linear dynamics, and offer significant challenges to classical control methods. Some of the key difficulties in these problems are that (i) It is often hard to write down, in closed form, a formal specification of the control task (for example, what is the objective function for "driving well"?), (ii) It is difficult to learn good control---as opposed to merely descriptive---models of the dynamics (cf. the "exploration problem" in reinforcement learning), and (iii) It is expensive to find closed-loop controllers for high dimensional, highly stochastic domains. In this talk, I will present formal results showing how these problems can be efficiently addressed in the apprenticeship learning setting, in which expert demonstrations of the task are available. I will also present an application of our ideas to autonomous helicopter flight and four-legged locomotion. Our helicopter results significantly extend the state of the art in helicopter control, and include the first successful completion of the following five aerobatic flight maneuvers: tic-toc, in-place forward flip and sideways roll, nose-in funnel, and tail-in funnel. Our resulting robot dog controller constitutes the state of the art in quadruped locomotion across rugged, previously unseen terrains.

About the Speaker: Pieter Abbeel is a PhD student in Prof. Andrew Ng's group at Stanford University. His research interests include machine learning, robotics, and control.