Seminar Series Archive

The duality between optimal control and Bayesian inference has been influential in designing feedback control systems in which perception and action are separable. In recent years, following progress in approximate inference, it was realized that this duality also holds in approximation and is applicable to the complete perception–action cycle, lending solid probabilistic interpretation to several successful algorithms for planning and learning in dynamical systems. We extend this framework by noting two useful capabilities of variational inference methods. First, the ability to infer latent variables can go beyond the interaction variables, namely the observations and actions, and can discover useful structure of the agent's internal memory process. Second, variational inference allows the proposal model to be different from the generative model, which enables the extraction of an acausal learning signal that is unused in previous work. We demonstrate these benefits in hierarchical imitation learning of control programs.

Roy Fox is an Assistant Professor and director of the Intelligent Dynamics Lab at the Department of Computer Science. He was previously a postdoc in UC Berkeley's BAIR, RISELab, and AUTOLAB, where he developed algorithms and systems that interact with humans to learn structured control policies for robotics and program synthesis. His research interests include theory and applications of reinforcement learning, control theory, information theory, and robotics. His current research focuses on structure, exploration, and optimization in deep reinforcement learning and imitation learning of virtual and physical agents.