Reinforcement Learning and Collusion by Clemens Possnig

Join us for another event in the Microeconomics Research Seminar Series, Summer Term 2024.

Clemens Possnig, from the University of Waterloo, will present their research on Reinforcement Learning and Collusion.

Abstract

This paper presents an analytical characterization of the long run policies learned by algorithms that interact repeatedly. The algorithms observe a state variable and update policies in order to maximize long term discounted payoffs. I show that the long run policies correspond to equilibria that are stable points of a tractable differential equation. As a running example, I consider a repeated Cournot game, for which learning the stage game Nash equilibrium serves as non-collusive benchmark. I give necessary and sufficient conditions for this Nash equilibrium not to be learned. These are requirements on the state variables algorithms use to determine their actions, and on the stage game. When algorithms determine actions based only on the past period's price, the Nash equilibrium can be learned. However, agents may condition their actions on richer types of information beyond the past period's price. In that case, I give sufficient conditions such that the policies converge with positive probability to a collusive equilibrium, while never converging to the Nash equilibrium. I show that such richer types of information can enable the learning of the payoff-maximal strongly symmetric equilibrium of the repeated game.

This seminar will be held in the Economics Common Room on Monday 22 April 2024 at 2.00pm. This event is open to all levels of study and is also open to the public.

To register your place and gain access to the webinar, please contact the seminar organisers.

This event is part of the Microeconomics Research Seminar Series.