Distinguished Hariri Institute/CISE Seminar: Asu Ozdaglar, Massachusetts Institute of Technology

Independent Learning Dynamics for Stochastic Games: Convergence and Finite-Time Analysis

Reinforcement learning (RL) has had tremendous successes in many artificial intelligence applications. Many of the forefront applications of RL involve multiple agents, e.g., playing chess and Go games, autonomous driving, and robotics. Unfortunately, classical RL framework is inappropriate for multi-agent learning as it assumes an agent’s environment is stationary and does not take into account the adaptive nature of opponent behavior. In this talk, I focus on stochastic games for multi-agent reinforcement learning in dynamic environments and develop independent learning dynamics for stochastic games: each agent is myopic and chooses best-response type actions to other agents’ strategies independently, meaning without any coordination with her opponents. There has been limited progress on developing convergent best-response type independent learning dynamics for stochastic games. I will present our recently proposed independent learning dynamics that guarantee convergence in zero-sum stochastic games. We then focus on the minimal information setting where agents do not observe opponent’s actions, but only observe the payoff they receive at each round. We present payoff-based and independent learning dynamics for such settings and provide finite-time guarantees using a novel coupled Lyapunov drift approach. In the end, I will present a new class of Markov games that models local payoff interactions in multi-agent stochastic games.

Asu Ozdaglar is the Mathworks Professor of Electrical Engineering and Computer Science (EECS) at the Massachusetts Institute of Technology (MIT). She is the department head of EECS and deputy dean of academics of the Schwarzman College of Computing at MIT. Her research expertise includes optimization, machine learning, economics, and networks. Her recent research focuses on designing incentives and algorithms for data-driven online systems with many diverse human-machine participants. She has investigated issues of data ownership and markets, spread of misinformation on social media, economic and financial contagion, and social learning. Professor Ozdaglar is the recipient of a Microsoft fellowship, the MIT Graduate Student Council Teaching award, the NSF Career award, the 2008 Donald P. Eckman award of the American Automatic Control Council, the 2014 Spira teaching award, and Keithley, Distinguished School of Engineering and Mathworks professorships. She is an IEEE fellow, IFAC fellow, and was selected as an invited speaker at the International Congress of Mathematicians. She received her Ph.D. degree in electrical engineering and computer science from MIT in 2003.

Faculty Host: Ayşe Coşkun

Student Host: Andres Chavez Armijos